feat: monky-deployd v0.1.0 — pull agent over the mesh (ADR-0028)

Stdlib-only Python 3.12 agent for docker VMs and laptops: flock → checkin
(bearer = the agent's OpenBao token, bootstrapped from the install kit's
jwt-tenancy deploy grant) → action apply|none|down → bundle (sha256
verified) → refusal checks (unresolved ${VAR} names only, manifest paths
pinned to monky/data/<env>/see/, privileged/host-network, rollback, disk
need×1.5+headroom) → lease → POST /v1/auth/jwt-tenancy/login → KV reads →
.env 0600 → promote → compose pull/up → wait healthy → report; finally
renew-self / re-lease before max TTL, scrub. Exit 0/75/78/1. Redactor log
filter. Transports sdk (openziti) / proxy (ziti tunnel proxy 18443/18200) /
system. Laptop mode.

Packaging: hardened oneshot + 60 s timer + proxy unit, nfpm .deb with
/opt/monky-deployd/venv, install.sh for Ubuntu 26.04 (Gitea release
download, enrol, ACLs, bootstrap from stdin), ansible role skeleton for
osg1-07. CI: lint/test on every change; wheel (openziti on ubuntu:26.04) and
package (nfpm) allow_failure until runner egress is proven; GitLab release +
release:gitea on v* tags. Docs: README, PROTOCOL, OPERATIONS, CHANGELOG,
CLAUDE/AGENTS.

Divergence noted: monky-tenancy main (MR !15) still ships the AppRole lease
and kit; this agent implements the plan's Gate 1 RESULT (login_jwt, no
unwrap) and refuses an AppRole lease loudly (LEASE_SHAPE).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
This commit is contained in:
2026-09-05 08:01:36 +00:00
parent c72d6c0227
commit 1c42e913a8
48 changed files with 4932 additions and 64 deletions
+10
View File
@@ -0,0 +1,10 @@
__pycache__/
*.pyc
.pytest_cache/
.ruff_cache/
build/
dist/
*.egg-info/
.cache/
vendor/*.whl
packaging/out/
+194
View File
@@ -0,0 +1,194 @@
# monky-deployd CI/CD (mirrors monky-tenancy's conventions)
#
# lint -> test -> build (openziti wheel) -> package (.deb via nfpm) -> release (v* tags) -> docs
#
# Every script line is single-quoted (a bare ": " turns the line into a YAML map and silently
# yields a 0-job pipeline). Jobs that need egress the runner may not have (GitHub for the
# openziti sdist's ziti-sdk-c fetch, GitHub for the nfpm binary) are allow_failure: true until
# proven; see README "CI notes".
#
# CI/CD variables (project or group level):
# GITEA_TOKEN — Gitea API token (write:repository) for `release:gitea`; without it the job is manual
# GITLAB_TRANSLATE_TOKEN — used by the translate-docs component (group level)
include:
# doc-translator via the CI/CD component; PARTIAL PIN @11.3 tracks 11.3.x (no `v`).
- component: $CI_SERVER_FQDN/tikali/platform/doc-translator/translate-docs@11.3
inputs:
docs_stage: docs
stages: [lint, test, build, package, release, docs]
# One pipeline per change: MR pipeline for merge requests, branch pipeline otherwise, never both.
workflow:
rules:
- if: '$CI_COMMIT_TAG'
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_PIPELINE_SOURCE == "push"'
when: never
- if: '$CI_COMMIT_BRANCH'
variables:
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
OPENZITI_VERSION: "1.7.1"
NFPM_VERSION: "2.43.0"
cache:
key: "$CI_JOB_NAME"
paths: [.cache/pip]
.run_rules:
rules:
- if: '$CI_COMMIT_TAG'
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH'
lint:
extends: [.run_rules]
stage: lint
image: python:3.12-slim
before_script:
- 'pip install -q "ruff>=0.16,<0.17"'
script:
- 'ruff check .'
- 'ruff format --check .'
- 'bash -n packaging/install.sh'
- 'bash -n packaging/scripts/postinstall.sh packaging/scripts/preremove.sh packaging/scripts/postremove.sh'
- 'python -c "import ast,sys; [ast.parse(open(f).read()) for f in sys.argv[1:]]" tests/fakebin/docker'
# the release version must match the tag when there is one
- 'if [ -n "$CI_COMMIT_TAG" ]; then v=$(python -c "import monky_deployd;print(monky_deployd.__version__)"); [ "v$v" = "$CI_COMMIT_TAG" ] || { echo "tag $CI_COMMIT_TAG != __version__ $v"; exit 1; }; fi'
test:
extends: [.run_rules]
stage: test
image: python:3.12-slim
before_script:
- 'pip install -q pytest'
script:
- 'chmod +x tests/fakebin/docker'
- 'pytest --junitxml=report.xml'
artifacts:
when: always
reports:
junit: report.xml
# --- the openziti wheel ---------------------------------------------------------------------
# PyPI ships `openziti` as an sdist whose build fetches ziti-sdk-c (+ prebuilt tlsuv/uv-mbed
# via cmake FetchContent) from github.com at install time. Building it here on ubuntu:26.04
# (the target OS; python3 = the target's python3) gives us a wheel to vendor into the venv.
# NEEDS runner egress to github.com + pypi.org; allow_failure until proven on this runner —
# without the wheel the .deb still builds (transports proxy/system work; sdk logs a clear error).
wheel:
stage: build
image: ubuntu:26.04
needs: ["test"]
allow_failure: true
rules:
- if: '$CI_COMMIT_TAG'
- if: '$CI_COMMIT_BRANCH == "main"'
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: manual
allow_failure: true
variables:
DEBIAN_FRONTEND: noninteractive
script:
- 'apt-get update -qq && apt-get install -y -qq --no-install-recommends python3 python3-venv python3-dev build-essential cmake ninja-build git pkg-config libssl-dev zlib1g-dev ca-certificates curl >/dev/null'
- 'python3 -m venv /tmp/wb && /tmp/wb/bin/pip install -q --upgrade pip wheel setuptools'
- 'mkdir -p vendor'
- '/tmp/wb/bin/pip wheel --no-deps --no-binary openziti "openziti==${OPENZITI_VERSION}" -w vendor/'
- 'ls -l vendor/'
artifacts:
paths: [vendor/*.whl]
expire_in: 30 days
# --- the .deb --------------------------------------------------------------------------------------
# venv at its final path (/opt/monky-deployd/venv is where the .deb puts it; venvs are not
# relocatable) + nfpm. nfpm comes from GitHub releases (egress) with the goreleaser apt repo as
# fallback; allow_failure until proven.
package:
stage: package
image: ubuntu:26.04
needs:
- job: test
- job: wheel
optional: true
allow_failure: true
rules:
- if: '$CI_COMMIT_TAG'
- if: '$CI_COMMIT_BRANCH == "main"'
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: manual
allow_failure: true
variables:
DEBIAN_FRONTEND: noninteractive
script:
- 'apt-get update -qq && apt-get install -y -qq --no-install-recommends python3 python3-venv python3-pip ca-certificates curl gnupg >/dev/null'
- 'VERSION=$(python3 -c "import monky_deployd;print(monky_deployd.__version__)"); echo "VERSION=$VERSION" | tee build.env'
- 'python3 -m venv /opt/monky-deployd/venv'
- '/opt/monky-deployd/venv/bin/pip install -q --upgrade pip'
- '/opt/monky-deployd/venv/bin/pip install -q .'
- 'if ls vendor/*.whl >/dev/null 2>&1; then /opt/monky-deployd/venv/bin/pip install -q vendor/*.whl && /opt/monky-deployd/venv/bin/python -c "import openziti; print(\"openziti\", openziti.__version__ if hasattr(openziti, \"__version__\") else \"ok\")"; else echo "WARNING: no vendored openziti wheel — transport sdk will not work from this build"; fi'
- '/opt/monky-deployd/venv/bin/python -m monky_deployd version'
- 'mkdir -p build dist && cp -a /opt/monky-deployd/venv build/venv'
# nfpm: GitHub release .deb, else the goreleaser apt repo
- 'curl -fsSL -o /tmp/nfpm.deb "https://github.com/goreleaser/nfpm/releases/download/v${NFPM_VERSION}/nfpm_${NFPM_VERSION}_amd64.deb" && apt-get install -y -qq /tmp/nfpm.deb >/dev/null || { echo "deb [trusted=yes] https://repo.goreleaser.com/apt/ /" > /etc/apt/sources.list.d/goreleaser.list; apt-get update -qq; apt-get install -y -qq nfpm; }'
- 'VERSION=$VERSION nfpm package --config packaging/nfpm.yaml --packager deb --target dist/'
- 'cd dist && for f in *.deb; do sha256sum "$f" > "$f.sha256"; done && ls -l && cd ..'
- 'cp packaging/install.sh dist/install.sh'
artifacts:
paths: [dist/]
reports:
dotenv: build.env
expire_in: 90 days
# --- GitLab release (v* tags): generic package registry + release with asset links -------------
release:
stage: release
image:
name: registry.gitlab.com/gitlab-org/release-cli:latest
entrypoint: [""]
needs: ["package"]
rules:
- if: '$CI_COMMIT_TAG =~ /^v/'
script:
- 'apk add --no-cache curl >/dev/null 2>&1 || true'
- 'VERSION=${CI_COMMIT_TAG#v}; PKG="${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/monky-deployd/${VERSION}"'
- 'for f in dist/*.deb dist/*.sha256 dist/install.sh; do curl -fsS --header "JOB-TOKEN: $CI_JOB_TOKEN" --upload-file "$f" "$PKG/$(basename "$f")"; echo; done'
- 'DEB=$(basename dist/*.deb)'
- >-
release-cli create --name "monky-deployd $CI_COMMIT_TAG" --tag-name "$CI_COMMIT_TAG"
--description "See CHANGELOG.md. Public install assets are on the Gitea mirror: https://gitea.cbs.tikali.net/mdella/monky-deployd/releases/tag/$CI_COMMIT_TAG"
--assets-link "{\"name\":\"$DEB\",\"url\":\"$PKG/$DEB\",\"link_type\":\"package\"}"
--assets-link "{\"name\":\"$DEB.sha256\",\"url\":\"$PKG/$DEB.sha256\",\"link_type\":\"other\"}"
--assets-link "{\"name\":\"install.sh\",\"url\":\"$PKG/install.sh\",\"link_type\":\"other\"}"
# --- Gitea release (the PUBLIC download the installer uses) ------------------------------------------
# The GitLab project is private, so install.sh fetches from the Gitea mirror
# https://gitea.cbs.tikali.net/mdella/monky-deployd/releases/download/v<ver>/... . This job
# waits for the pull-mirror to carry the tag, creates the release and uploads the assets.
# Automatic when GITEA_TOKEN is set; otherwise a manual, non-blocking job (docs/OPERATIONS.md
# has the by-hand recipe).
release:gitea:
stage: release
image:
name: alpine:3.20
entrypoint: [""]
needs: ["package"]
rules:
- if: '$CI_COMMIT_TAG =~ /^v/ && $GITEA_TOKEN'
- if: '$CI_COMMIT_TAG =~ /^v/'
when: manual
allow_failure: true
variables:
GITEA_API: "https://gitea.cbs.tikali.net/api/v1/repos/mdella/monky-deployd"
script:
- 'apk add --no-cache curl jq >/dev/null'
- '[ -n "$GITEA_TOKEN" ] || { echo "GITEA_TOKEN is not set"; exit 1; }'
- 'H="Authorization: token $GITEA_TOKEN"'
- 'curl -fsS -X POST -H "$H" "$GITEA_API/mirror-sync" >/dev/null || echo "mirror-sync trigger failed; polling anyway"'
- 'for i in $(seq 1 30); do curl -fsS -H "$H" "$GITEA_API/tags/$CI_COMMIT_TAG" >/dev/null 2>&1 && break; echo "waiting for the mirror to carry $CI_COMMIT_TAG ($i)"; sleep 10; done'
- 'curl -fsS -H "$H" "$GITEA_API/tags/$CI_COMMIT_TAG" >/dev/null || { echo "tag not on the mirror yet"; exit 1; }'
- 'RID=$(curl -fsS -H "$H" "$GITEA_API/releases/tags/$CI_COMMIT_TAG" 2>/dev/null | jq -r .id || true)'
- 'if [ -z "$RID" ] || [ "$RID" = "null" ]; then RID=$(curl -fsS -X POST -H "$H" -H "Content-Type: application/json" "$GITEA_API/releases" -d "{\"tag_name\":\"$CI_COMMIT_TAG\",\"name\":\"monky-deployd $CI_COMMIT_TAG\",\"body\":\"See CHANGELOG.md\",\"draft\":false,\"prerelease\":false}" | jq -r .id); fi'
- 'echo "release id $RID"'
- 'for f in dist/*.deb dist/*.sha256 dist/install.sh; do n=$(basename "$f"); curl -fsS -X POST -H "$H" -F "attachment=@$f" "$GITEA_API/releases/$RID/assets?name=$n" >/dev/null && echo "uploaded $n"; done'
+31
View File
@@ -0,0 +1,31 @@
<!-- xlate:verbatim-fences -->
# AGENTS.md — rules for AI agents working in monky-deployd
Read [CLAUDE.md](CLAUDE.md) first; these are the non-negotiables.
1. **Stdlib only.** No runtime dependency in `monky_deployd/`; `openziti` is optional and imported
lazily in `transport.SdkTransport` only.
2. **Never a secret value in a log, a report, `state.json`, an exception message or a test
assertion output.** Register every value/token/grant with `REDACTOR.add()` the moment it exists;
name variables, never values. `tests/test_agent.py` asserts the report tail is clean — keep it so.
3. **Auth is the jwt-tenancy deploy grant** (`login_jwt``POST /v1/auth/jwt-tenancy/login`). No
AppRole, no unwrap, no compatibility fallback; an AppRole-shaped lease is `LEASE_SHAPE` (failed).
4. **Vocabulary:** check-in `action apply|none|down`; report `result applied|failed|down`.
`env_id`, backend, bundle, deploy grant. Never "tenant".
5. **Refuse before you lease**, and refuse hard: a failed check is `Refusal(code, detail)` → report
`failed` → exit 1. No "apply anyway with a warning".
6. **Exit codes 0 / 75 / 78 / 1 are a contract** (timer `SuccessExitStatus=75`; 78 stops the loop).
7. **Prod never purges volumes**, whatever tenancy sends.
8. **Config is the YAML subset** `config.parse_yaml_subset` understands (maps, scalars, simple lists).
If you add a key: `Config` dataclass + `config.example.yaml` + `install.sh` + the ansible template.
9. **Tests are hermetic** (fake tenancy/OpenBao HTTP servers + the stub `docker`). Every new refusal
code or protocol field gets a test against the fakes, and the fakes track the tenancy shapes of
record (monky-tenancy `app/schemas_backends.py`, design doc 24 §3.3).
10. **Quality gates, each its own statement:** `ruff format --check .`, `ruff check .`, `pytest`,
`bash -n packaging/install.sh`, `systemd-analyze verify` where available.
11. **Branching:** `feat/*` / `fix/*` / `docs/*` → MR into `main` → tag `vX.Y.Z` on `main` (protected
`v*`). The version lives in `monky_deployd/__init__.py`, `pyproject.toml`, `install.sh`
`DEFAULT_VERSION`, `CHANGELOG.md` — bump all four.
12. **YAML traps:** single-quote every CI script line; never `": "` in an unquoted scalar.
13. **Never edit a `*.ru.md`**; edit English only, keep `<!-- xlate:verbatim-fences -->` on line 1.
14. **ADR citations are `MONKY-ADR-NNNN`** (monky-design-docs); cite, don't restate.
+21
View File
@@ -0,0 +1,21 @@
<!-- xlate:verbatim-fences -->
# Changelog
## v0.1.0 — 2026-09-05
First release (MONKY-ADR-0028 §D, Reconciliation v2, Gate 1 v2).
- Stdlib-only Python 3.12 agent: `run --once|loop`, `status`, `bootstrap`, `--prune`.
- Protocol: `POST /v1/agent/checkin` (`action apply|none|down`), `GET /v1/agent/bundle/{env}/{sha}`
(sha256 verified), `POST /v1/agent/lease`**deploy grant** (`login_jwt`), `POST /v1/agent/report`
(`result applied|failed|down`, redacted `log_tail`). An AppRole-shaped lease is refused (`LEASE_SHAPE`).
- OpenBao: `POST /v1/auth/jwt-tenancy/login {"role":"see-env","jwt":…}`; KV-v2 reads pinned to the
manifest's versions, paths pinned to `monky/data/<env>/see/`; renew-self / re-lease before max TTL.
- Refusals: `ENV_INCOMPLETE` (names only), `PRIVILEGED_REFUSED`, `ROLLBACK_REFUSED`, `DISK_INSUFFICIENT`
(`need × 1.5 + 2 GiB` vs docker data-root), `BUNDLE_SHA_MISMATCH`, `BUNDLE_ENV_MISMATCH`.
- Transports `sdk` (openziti SDK), `proxy` (`ziti tunnel proxy` 18443/18200), `system`.
- Exit codes 0 / 75 / 78 (`AGENT_ENV_MISMATCH`, no retry storm) / 1; laptop mode (offline exits 0).
- Packaging: hardened `monky-deployd.service` oneshot + 60 s timer, `monky-deployd-proxy.service`,
`nfpm` `.deb` with `/opt/monky-deployd/venv`, `packaging/install.sh` (Ubuntu 26.04), ansible role skeleton.
- Known divergence: monky-tenancy `main` (MR !15) still ships the AppRole lease/kit; the JWT-grant
follow-up is the tenancy side of this release (docs/PROTOCOL.md "Divergences").
+90
View File
@@ -0,0 +1,90 @@
<!-- xlate:verbatim-fences -->
# monky-deployd — CLAUDE.md
## What this is
**monky-deployd** is the **on-box pull agent** for Monky backend environments on docker VMs and
laptops (MONKY-ADR-0028). It dials **monky-tenancy** over the OpenZiti mesh with the box's own host
identity, converges the box to the published bundle (`docker compose`), reads its secrets from
**OpenBao** itself, and reports — the check-in is also the liveness heartbeat. It is the apply half
of the backend lifecycle; the API/DB half is monky-tenancy (`/v1/backends*`, `/v1/agent/*`), the
composition source is monky-deploy (renderer), the console is monky-mgmt-ui. k8s backends have no
agent (ArgoCD applies).
## Vocabulary (hard rule)
**backend**, **environment** (`env_id` = `env-<tier>-<nn>`), **bundle**, **deploy grant**,
**check-in**. Check-in returns `action ∈ apply|none|down`; a report carries
`result ∈ applied|failed|down`. There is no `desired_sha="down"`, no "absent", no "up" result.
Never "tenant" (the API repo's name is the one exception).
## Non-negotiables
- **Stdlib only** in `monky_deployd/`. The only optional import is `openziti` (transport `sdk`),
loaded lazily inside `transport.SdkTransport`. No PyYAML: `config.parse_yaml_subset` reads the
config; keep the install kit / ansible template inside that subset.
- **The agent never receives a secret from tenancy** and **never logs a value**. Every value read
from OpenBao (and every token/grant) goes through `redact.REDACTOR.add()` the moment it exists;
refusals and reports name variables, never values. Tests assert the report tail carries none.
- **No AppRole, nothing to unwrap.** Auth is `POST /v1/auth/jwt-tenancy/login {"role":"see-env",
"jwt":<grant>}`; the resulting Bao token is the bearer to tenancy. An AppRole-shaped lease is a
loud `LEASE_SHAPE` failure, not a fallback.
- **Refuse before you lease.** All bundle checks (sha, env, unresolved vars, manifest paths,
privileged, rollback, disk) run before `POST /v1/agent/lease`, so a refused bundle costs no lease
(5/h budget) and no OpenBao login.
- **Exit codes are a contract** with the timer and the loop: 0 / 75 (temporary, retry) /
78 (`AGENT_ENV_MISMATCH`, never retried) / 1. `SuccessExitStatus=75` in the unit.
- **Prod never purges.** `-v` is dropped on prod even if tenancy asks; the agent never decides state.
- **Paths**: bearer at `/var/lib/monky-deployd/bao.token` (0600), grant at
`/etc/monky-deployd/bootstrap.jwt` (consumed then deleted/truncated), releases under
`<deploy_dir>/releases/<sha>` with a `current` symlink = the compose project directory.
## Layout
| module | role |
|---|---|
| `cli.py` | argparse, journald-friendly logging (`<N>` priority when `JOURNAL_STREAM`), `run/status/bootstrap/version` |
| `agent.py` | the tick (`Agent.run_once`), action dispatch, refusal checks, token upkeep, staging/promote |
| `config.py` | YAML-subset reader + `Config` dataclasses + validation |
| `transport.py` | `sdk` / `proxy` / `system` transports + `HttpClient` (http.client with TLS SNI) |
| `tenancy.py` | the four calls; error classes `Unauthenticated`, `EnvMismatch`, `RateLimited`, `LeaseShapeUnsupported` |
| `bao.py` | login, lookup/renew/revoke-self, KV-v2 read, `kv_data_path` (env pinning) |
| `bundle.py` | tar parse, `bundle_sha` (tenancy's formula), unresolved-var / privileged scans, `.env` rendering |
| `compose.py` | `docker compose` wrapper, `ps --format json` parsing (NDJSON + array), health wait |
| `state.py` | `state.json`, token file, atomic 0600 writes, `flock` |
| `redact.py` | the log filter |
## Tests
`pytest` is hermetic: `tests/fakes.py` runs a fake tenancy agent endpoint and a fake OpenBao as
real HTTP servers on loopback (shapes of record, incl. env pinning, superseded grants, rate
limits, the AppRole shape), and `tests/fakebin/docker` is a stub on `PATH` that records every
invocation. No network, no docker daemon, no root. Add a test for every new refusal code.
## Gates before you push
```sh
ruff format --check . || exit 1
ruff check . || exit 1
pytest || exit 1
bash -n packaging/install.sh || exit 1
systemd-analyze verify packaging/systemd/*.service # when available
```
## Releasing
Bump `monky_deployd.__version__` + `pyproject.toml` + `packaging/install.sh` `DEFAULT_VERSION` +
`CHANGELOG.md`, merge to `main`, tag `vX.Y.Z` (protected `v*`). The tag pipeline builds the `.deb`,
publishes the GitLab release and — with `GITEA_TOKEN` — the **Gitea release** the installer
downloads from (`docs/OPERATIONS.md` has the manual recipe). `lint` refuses a tag whose version
differs from `__version__`.
## What NOT to do
- Don't add a dependency to the agent; don't import `openziti` at module top level.
- Don't log, print or report a secret value, a token or a grant; don't put values in `state.json`.
- Don't add an AppRole/unwrap path "for compatibility".
- Don't apply when a check fails "with a warning"; refuse, report, exit 1.
- Don't retry `AGENT_ENV_MISMATCH`; don't turn 75 into a busy loop (the timer is the retry).
- Don't edit `*.ru.md` (doc-translator owns them); keep line 1 `<!-- xlate:verbatim-fences -->`.
- Cite decisions as `MONKY-ADR-NNNN`; don't restate them here.
+122 -64
View File
@@ -1,93 +1,151 @@
<!-- xlate:verbatim-fences -->
# monky-deployd
## Getting started
To make it easy for you to get started with GitLab, here's a list of recommended next steps.
Already a pro? Just edit this README.md and make it your own. Want to make it easy? [Use the template at the bottom](#editing-this-readme)!
## Add your files
* [Create](https://docs.gitlab.com/user/project/repository/web_editor/#create-a-file) or [upload](https://docs.gitlab.com/user/project/repository/web_editor/#upload-a-file) files
* [Add files using the command line](https://docs.gitlab.com/topics/git/add_files/#add-files-to-a-git-repository) or push an existing Git repository with the following command:
**The on-box pull agent for Monky backends** ([MONKY-ADR-0028](https://scm.tikali.ai/tikali/applications/monky/monky-design-docs/-/blob/develop/adr/0028-backend-lifecycle-ownership-and-execution.md), [design doc 24](https://scm.tikali.ai/tikali/applications/monky/monky-design-docs/-/blob/develop/docs/24-backend-lifecycle.md)).
A docker VM or a laptop that hosts a backend environment runs this agent; every minute it dials
**monky-tenancy over the OpenZiti mesh with the box's own host identity**, asks what the backend
should be running, and converges: fetch the rendered bundle, lease a deploy grant, log in to
OpenBao, read its own secrets, `docker compose up`, report. Nothing pushes into the box, no SSH,
no credentials for the box are held anywhere else. The same check-in is the liveness heartbeat.
```
cd existing_repo
git remote add origin https://scm.tikali.ai/tikali/applications/monky/monky-deployd.git
git branch -M main
git push -uf origin main
box / laptop ◄── mesh ──► monky-deployd: checkin → bundle → lease → OpenBao → compose up → report
```
## Integrate with your tools
Python 3.12+, **stdlib only** (the `openziti` SDK is optional and vendored into the `.deb`).
Verified on **Ubuntu 26.04**.
* [Set up project integrations](https://scm.tikali.ai/tikali/applications/monky/monky-deployd/-/settings/integrations)
## Install (one-liner, from the enrolment kit)
## Collaborate with your team
An admin reveals the kit once in the console (`GET /v1/backends/{id}/agent/install`); it hands
you the enrolment JWT and a one-time **bootstrap deploy grant**. On the box:
* [Invite team members and collaborators](https://docs.gitlab.com/user/project/members/)
* [Create a new merge request](https://docs.gitlab.com/user/project/merge_requests/creating_merge_requests/)
* [Automatically close issues from merge requests](https://docs.gitlab.com/user/project/issues/managing_issues/#closing-issues-automatically)
* [Enable merge request approvals](https://docs.gitlab.com/user/project/merge_requests/approvals/)
* [Set auto-merge](https://docs.gitlab.com/user/project/merge_requests/auto_merge/)
```sh
curl -fsSL https://gitea.cbs.tikali.net/mdella/monky-deployd/raw/branch/main/packaging/install.sh \
| sudo bash -s -- --env env-qa-02 --site cbs --enrol-jwt ./monky-host.env-qa-02.jwt < bootstrap.jwt
# [--transport sdk|proxy|system] [--version 0.1.0] [--laptop] [--bao-ca openbao-ca.pem]
```
## Test and Deploy
`install.sh` installs `ziti-edge-tunnel` (OpenZiti `jammy` suite) and `docker-compose-plugin` if
absent, downloads the pinned `.deb` + `.sha256` from the [Gitea release](https://gitea.cbs.tikali.net/mdella/monky-deployd/releases),
enrols `monky-host.<env>` if the identity is missing, switches the tunneler to `run-host`,
writes `/etc/monky-deployd/config.yaml`, grants the agent read access to the identity (ACL),
stages the bootstrap grant (0600), enables `monky-deployd.timer`, runs one tick and deletes the
JWT. The GitLab project is private, so **the public download is the Gitea mirror**.
Use the built-in continuous integration in GitLab.
## Transports
* [Get started with GitLab CI/CD](https://docs.gitlab.com/ci/quick_start/)
* [Analyze your code for known vulnerabilities with Static Application Security Testing (SAST)](https://docs.gitlab.com/user/application_security/sast/)
* [Deploy to Kubernetes, Amazon EC2, or Amazon ECS using Auto Deploy](https://docs.gitlab.com/topics/autodevops/requirements/)
* [Use pull-based deployments for improved Kubernetes management](https://docs.gitlab.com/user/clusters/agent/)
* [Set up protected environments](https://docs.gitlab.com/ci/environments/protected_environments/)
| `transport` | how | when |
|---|---|---|
| `sdk` (default) | the OpenZiti Python SDK dials `monky.tenancy.deploy` / `openbao` by service name with the same host identity `ziti-edge-tunnel run-host` uses; no tun, no root beyond the docker group | VMs and laptops with the vendored wheel |
| `proxy` | `monky-deployd-proxy.service` runs `ziti tunnel proxy -i <identity> monky.tenancy.deploy:18443 openbao:18200` as user `ziti`; the agent talks to `127.0.0.1:18443/18200` (TLS SNI + cert check still `bao.cbs.tikali.net`) | the wheel is unavailable; `ziti` CLI present |
| `system` | plain DNS/TCP | laptops whose tunneler runs in `run` mode (tun + DNS) |
***
## Commands and exit codes
# Editing this README
```
monky-deployd run --once [--prune] # one tick (what the timer runs); --prune drops old releases + docker image prune
monky-deployd run # loop (laptop mode: no timer); SIGTERM stops it
monky-deployd status [--json] # token present? applied vs desired sha, last checkin/report, compose ps
monky-deployd bootstrap [--force] # log in to OpenBao with /etc/monky-deployd/bootstrap.jwt, store the token
monky-deployd version
```
When you're ready to make this README your own, just edit this file and use the handy template below (or feel free to structure it however you want - this is just a starting point!). Thanks to [makeareadme.com](https://www.makeareadme.com/) for this template.
| exit | meaning |
|---|---|
| `0` | converged / heartbeat sent — or offline in `laptop_mode` |
| `75` | temporary network failure (mesh down, tenancy unreachable, `429`); the next tick retries — `SuccessExitStatus=75` in the unit |
| `78` | `AGENT_ENV_MISMATCH`: the token is pinned to another environment than `config.yaml` says; **not retried** (loop mode exits) |
| `1` | refusal or failure (`ENV_INCOMPLETE`, `PRIVILEGED_REFUSED`, `DISK_INSUFFICIENT`, `ROLLBACK_REFUSED`, `BUNDLE_SHA_MISMATCH`, compose failure, `AGENT_UNAUTHENTICATED`, no credentials) — reported to tenancy as `failed` with the code and secret *names* only |
## Suggestions for a good README
## What a tick does
Every project is different, so consider which of these sections apply to yours. The sections used in the template are suggestions for most open source projects. Also keep in mind that while a README can be too long and detailed, too long is better than too short. If you think your README is too long, consider utilizing another form of documentation rather than cutting out information.
1. `flock` (a second concurrent tick exits 0) → load `state.json`.
2. Bearer = the OpenBao token at `/var/lib/monky-deployd/bao.token` (0600). Absent → log in to
`jwt-tenancy` with the kit's bootstrap grant (`/etc/monky-deployd/bootstrap.jwt`), then delete it.
3. `POST /v1/agent/checkin` `{env_id, agent_version, applied_sha, host, containers}``action`:
- **`down`** → `docker compose down --remove-orphans` (`-v` only if `purge_volumes` was
requested or `volumes_on_absent: purge`, **never on prod**) → report `down`.
- **`none`** → `compose ps`; healthy → report `applied` (heartbeat); unhealthy → report `failed`.
- **`apply`** → `GET /v1/agent/bundle/{env}/{sha}`, verify **sha256** over the files, then refuse on:
unresolved `${VAR}` (names only), manifest paths outside `monky/data/<env>/see/`, `privileged`/host
network/`SYS_ADMIN` unless `bundle.json` `allow_privileged`, a rollback unless `allow_rollback`,
docker data-root free space `< need × 1.5 + 2 GiB``POST /v1/agent/lease` → OpenBao
`POST /v1/auth/jwt-tenancy/login {"role":"see-env","jwt":<grant>}` → KV reads per
`secrets.manifest.json` (pinned versions) → `.env` written atomically 0600 into a staging dir →
promoted to `releases/<sha>` + `current``compose pull``up -d --remove-orphans` → wait
healthy → report `applied` (or `failed` with the compose log tail).
4. `finally`: renew-self when the TTL runs low, re-lease before max TTL, scrub secrets from memory,
remove staging dirs, save state.
## Name
Choose a self-explaining name for your project.
## Security model
## Description
Let people know what your project can do specifically. Provide context and add a link to any reference visitors might be unfamiliar with. A list of Features or a Background subsection can also be added here. If there are alternatives to your project, this is a good place to list differentiating factors.
- **Identity = the box's ziti host identity.** Only identities with `#monky-deploy-agent` can dial
tenancy's agent entrypoint; `#openbao-client` reaches OpenBao. The agent reads the identity
through an ACL (`setfacl -m u:monky-deployd:r`), never owns it.
- **The bearer to tenancy is the agent's own OpenBao token**, minted by OpenBao from a
tenancy-signed ES256 deploy grant (`aud openbao-see-env`, `kind deploy-grant`, 1 h, single-use
`jti`). Tenancy verifies it with `auth/token/lookup`, pins `meta.env_id`, and refuses a token
whose `meta.grant_jti` was superseded (kit re-reveal, retire) → `401 AGENT_UNAUTHENTICATED`.
**No AppRole, nothing to unwrap** (Gate 1 result, 2026-09-05).
- **The agent never receives a secret from tenancy.** Bundles carry placeholders; the agent reads
`monky/data/<env_id>/see/*` itself, and the OpenBao policy is templated on the token's entity
(one entity per env), so another env's path is a 403 — and refused locally before any read.
- **Journald never carries a value.** A `Redactor` filter masks token shapes (`hvs.*`, JWTs,
`Authorization: Bearer`) and every value the agent has read; the report's `log_tail` goes through
the same scrubber. Secret *names* are logged.
- **Hardened oneshot**: user `monky-deployd` (+ `docker` group), `NoNewPrivileges`,
`ProtectSystem=strict`, `ReadWritePaths` only the state dir, `/etc/monky-deployd` (to consume the
grant) and the docker socket, `UMask=0077`, no capabilities.
- **Prod is special**: `-v` is never passed on a prod env, and tenancy never auto-changes prod state.
## Badges
On some READMEs, you may see small images that convey metadata, such as whether or not all the tests are passing for the project. You can use Shields to add some to your README. Many services also have instructions for adding a badge.
## Ubuntu 26.04 verified checklist (from design doc 24 §2.1a / plan §D 2.8)
## Visuals
Depending on what you are making, it can be a good idea to include screenshots or even a video (you'll frequently see GIFs rather than actual videos). Tools like ttygif can help, but check out Asciinema for a more sophisticated method.
```
python3 --version # >= 3.12 (26.04 ships 3.14)
docker compose version # compose plugin
systemctl is-active ziti-edge-tunnel # run-host (drop-in run-host.conf)
ziti tunnel proxy --help # fallback transport present (transport proxy)
/opt/monky-deployd/venv/bin/python -c 'import openziti' # SDK import (transport sdk)
monky-deployd status # token present; applied == desired
journalctl -u monky-deployd -n 50 # "checkin: action=..." then "applied ..."
df -h $(docker info -f '{{.DockerRootDir}}') # free >= bundle need x 1.5 + headroom
docker compose -p monky-<env> ps # healthy
curl monky.percept.<env>:47283/health # from another mesh member -> 200
```
## Installation
Within a particular ecosystem, there may be a common way of installing things, such as using Yarn, NuGet, or Homebrew. However, consider the possibility that whoever is reading your README is a novice and would like more guidance. Listing specific steps helps remove ambiguity and gets people to using your project as quickly as possible. If it only runs in a specific context like a particular programming language version or operating system or has dependencies that have to be installed manually, also add a Requirements subsection.
## Repository layout
## Usage
Use examples liberally, and show the expected output if you can. It's helpful to have inline the smallest example of usage that you can demonstrate, while providing links to more sophisticated examples if they are too long to reasonably include in the README.
```
monky_deployd/ cli.py config.py transport.py tenancy.py bao.py bundle.py compose.py state.py redact.py agent.py
packaging/ install.sh systemd/{monky-deployd.service,.timer,monky-deployd-proxy.service} nfpm.yaml scripts/
ansible/roles/monky_deployd/ role skeleton for osg1-07 (env-dev-06..09 rollout)
docs/ PROTOCOL.md (the four calls + the grant flow) OPERATIONS.md (systemd, logs, retire, laptops)
tests/ hermetic: fake tenancy + fake OpenBao HTTP servers, a stub `docker` on PATH
```
## Support
Tell people where they can go to for help. It can be any combination of an issue tracker, a chat room, an email address, etc.
## Development
## Roadmap
If you have ideas for releases in the future, it is a good idea to list them in the README.
```sh
ruff check . && ruff format --check .
pytest # no network, no docker: fakes only
bash -n packaging/install.sh
systemd-analyze verify packaging/systemd/*.service # where systemd is available
```
## Contributing
State if you are open to contributions and what your requirements are for accepting them.
### CI notes
For people who want to make changes to your project, it's helpful to have some documentation on how to get started. Perhaps there is a script that they should run or some environment variables that they need to set. Make these steps explicit. These instructions could also be useful to your future self.
`lint` and `test` run on every MR/branch. `wheel` builds the `openziti` wheel on `ubuntu:26.04`
(PyPI ships an sdist that fetches **ziti-sdk-c from github.com** at build time — the runner needs
egress to github.com and pypi.org) and `package` builds the `.deb` with `nfpm` (binary from GitHub
releases, goreleaser apt repo as fallback). Both are `allow_failure: true` until proven on this
runner; without the wheel the `.deb` still works with `transport: proxy|system`. On a `v*` tag
`release` uploads to the GitLab generic package registry + release, and `release:gitea` publishes
the same assets on the public Gitea mirror (automatic when `GITEA_TOKEN` is set, manual otherwise
— see `docs/OPERATIONS.md` for the by-hand recipe).
You can also document commands to lint the code or run tests. These steps help to ensure high code quality and reduce the likelihood that the changes inadvertently break something. Having instructions for running tests is especially helpful if it requires external setup, such as starting a Selenium server for testing in a browser.
## See also
## Authors and acknowledgment
Show your appreciation to those who have contributed to the project.
## License
For open source projects, say how it is licensed.
## Project status
If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers.
- `docs/PROTOCOL.md`, `docs/OPERATIONS.md`, `CHANGELOG.md`
- monky-tenancy `docs/usage.md` (agent protocol), `app/api/agent.py`, `app/schemas_backends.py`
- monky-deploy (the renderer whose `render_files` produces the bundle)
+36
View File
@@ -0,0 +1,36 @@
# role `monky_deployd`
Installs and configures [monky-deployd](https://scm.tikali.ai/tikali/applications/monky/monky-deployd)
(the Monky backend pull agent, MONKY-ADR-0028) on a docker host that already carries an enrolled
host identity (`roles/ziti_tunneler`, run-host mode). Skeleton for **osg1-07**; copy it there.
What it does: pin + download the `.deb` from the Gitea release (sha256 verified) → ACL
`u:monky-deployd:r` on the identity (`rx` on the dir) → `/etc/monky-deployd/config.yaml` from the
template → optional openbao-ca PEM → optional `monky-deployd-proxy.service` (`transport: proxy`)
→ the **one-time bootstrap deploy grant** from a vault var (0600, `no_log`) → `monky-deployd.timer`
→ first tick via handler (inside the grant's hour) → `monky-deployd status`.
## Variables (see `defaults/main.yml`)
| var | note |
|---|---|
| `monky_deployd_version` | pinned release, e.g. `0.1.0` |
| `monky_deployd_env_id` / `_site` / `_transport` | per host (`env-dev-06`, `cbs`, `sdk`) |
| `monky_deployd_bootstrap_grant` | tenancy-minted deploy grant (1 h) — `ansible-vault` or a lookup at play time; empty keeps the existing token |
| `monky_deployd_bao_ca_pem` | the `openbao-ca` certificate (PEM) |
| `monky_deployd_volumes_on_absent` | `keep` (default) or `purge` (never applied on prod by the agent) |
## Example play
```yaml
- hosts: env-dev-06:env-dev-07:env-dev-08:env-dev-09
become: true
roles:
- role: ziti_tunneler # enrol/verify monky-host.<env>; gains the ACL var + name assertion
- role: monky_deployd
vars:
monky_deployd_bootstrap_grant: "{{ lookup('pipe', 'tenancy-mint-grant ' ~ inventory_hostname) }}"
```
Rollout order (plan §E): pilot env-qa-02 → env-dev-06..09 → env-dev-01 last (after its live key moves into Bao).
Verified on Ubuntu 26.04.
@@ -0,0 +1,35 @@
---
# monky_deployd — install and configure the Monky backend pull agent (MONKY-ADR-0028 §D).
# Copy this role into osg1-07 (roles/monky_deployd) and roll to env-dev-06..09 after the pilot.
monky_deployd_version: "0.1.0"
monky_deployd_base_url: "https://gitea.cbs.tikali.net/mdella/monky-deployd"
monky_deployd_deb: "monky-deployd_{{ monky_deployd_version }}_amd64.deb"
monky_deployd_deb_url: "{{ monky_deployd_base_url }}/releases/download/v{{ monky_deployd_version }}/{{ monky_deployd_deb }}"
monky_deployd_deb_sha256_url: "{{ monky_deployd_deb_url }}.sha256"
# per host (inventory / host_vars)
monky_deployd_env_id: "{{ inventory_hostname }}" # env-dev-06 ...
monky_deployd_site: cbs
monky_deployd_transport: sdk # sdk | proxy | system
monky_deployd_identity: "/opt/openziti/etc/identities/monky-host.{{ monky_deployd_env_id }}.json"
monky_deployd_laptop_mode: false
monky_deployd_volumes_on_absent: keep
monky_deployd_interval_s: 60
monky_deployd_healthy_timeout_s: 300
monky_deployd_disk_factor: 1.5
monky_deployd_disk_headroom_bytes: 2147483648
monky_deployd_tenancy_service: monky.tenancy.deploy
monky_deployd_tenancy_port: 8081
monky_deployd_bao_addr: "https://bao.cbs.tikali.net:8200"
monky_deployd_bao_auth_mount: jwt-tenancy
monky_deployd_bao_role: see-env
monky_deployd_bao_kv_mount: monky
# the openbao-ca certificate (PEM text) — from the vault (ansible-vault / OpenBao lookup)
monky_deployd_bao_ca_pem: ""
# ONE-TIME bootstrap deploy grant, minted by tenancy at playbook time (1 h): the role stages it
# and runs the first tick inside the hour. Leave empty to keep an existing bao.token.
monky_deployd_bootstrap_grant: ""
monky_deployd_run_first_tick: true
monky_deployd_timer_enabled: true
@@ -0,0 +1,6 @@
---
- name: monky_deployd tick
ansible.builtin.systemd:
name: monky-deployd.service
state: started
when: monky_deployd_run_first_tick | bool
+13
View File
@@ -0,0 +1,13 @@
---
galaxy_info:
role_name: monky_deployd
author: Tikali platform
description: Monky backend pull agent (monky-deployd) on docker hosts
license: Proprietary
min_ansible_version: "2.15"
platforms:
- name: Ubuntu
versions: ["noble", "26.04"]
collections:
- ansible.posix
dependencies: []
+130
View File
@@ -0,0 +1,130 @@
---
# Assumes: docker + compose plugin present, the host identity enrolled by roles/ziti_tunneler
# (run-host mode) — that role gains the ACL var + a name assertion (plan §D).
- name: monky_deployd | assert inputs
ansible.builtin.assert:
that:
- monky_deployd_env_id is match('^env-(dev|qa|stage|prod)-[0-9]{2,3}$')
- monky_deployd_site in ['cbs', 'pdx']
- monky_deployd_transport in ['sdk', 'proxy', 'system']
fail_msg: "env_id/site/transport out of grammar"
- name: monky_deployd | prerequisites
ansible.builtin.apt:
name: [acl, ca-certificates]
state: present
update_cache: true
cache_valid_time: 3600
- name: monky_deployd | installed version
ansible.builtin.command: dpkg-query -W -f='${Version}' monky-deployd
register: monky_deployd_installed
changed_when: false
failed_when: false
- name: monky_deployd | download .deb + sha256 from the Gitea release
when: monky_deployd_installed.stdout != monky_deployd_version
block:
- name: monky_deployd | fetch sha256
ansible.builtin.uri:
url: "{{ monky_deployd_deb_sha256_url }}"
return_content: true
register: monky_deployd_sha
- name: monky_deployd | fetch .deb (checksum verified)
ansible.builtin.get_url:
url: "{{ monky_deployd_deb_url }}"
dest: "/var/cache/apt/archives/{{ monky_deployd_deb }}"
checksum: "sha256:{{ monky_deployd_sha.content.split()[0] }}"
mode: "0644"
- name: monky_deployd | install .deb
ansible.builtin.apt:
deb: "/var/cache/apt/archives/{{ monky_deployd_deb }}"
- name: monky_deployd | identity present
ansible.builtin.stat:
path: "{{ monky_deployd_identity }}"
register: monky_deployd_id_stat
failed_when: not monky_deployd_id_stat.stat.exists
- name: monky_deployd | ACL so the agent can read the identity
ansible.posix.acl:
path: "{{ item.path }}"
entity: monky-deployd
etype: user
permissions: "{{ item.perms }}"
state: present
loop:
- { path: "{{ monky_deployd_identity }}", perms: r }
- { path: "{{ monky_deployd_identity | dirname }}", perms: rx }
- name: monky_deployd | directories
ansible.builtin.file:
path: "{{ item.path }}"
state: directory
owner: "{{ item.owner }}"
group: monky-deployd
mode: "{{ item.mode }}"
loop:
- { path: /etc/monky-deployd, owner: root, mode: "0750" }
- { path: /var/lib/monky-deployd, owner: monky-deployd, mode: "0700" }
- name: monky_deployd | openbao CA
ansible.builtin.copy:
content: "{{ monky_deployd_bao_ca_pem }}"
dest: /etc/monky-deployd/openbao-ca.pem
mode: "0644"
when: monky_deployd_bao_ca_pem | length > 0
- name: monky_deployd | config.yaml
ansible.builtin.template:
src: config.yaml.j2
dest: /etc/monky-deployd/config.yaml
owner: root
group: monky-deployd
mode: "0640"
notify: monky_deployd tick
- name: monky_deployd | proxy transport env
ansible.builtin.copy:
content: "ZITI_IDENTITY={{ monky_deployd_identity }}\n"
dest: /etc/monky-deployd/proxy.env
mode: "0644"
when: monky_deployd_transport == 'proxy'
- name: monky_deployd | proxy service
ansible.builtin.systemd:
name: monky-deployd-proxy.service
enabled: "{{ monky_deployd_transport == 'proxy' }}"
state: "{{ 'started' if monky_deployd_transport == 'proxy' else 'stopped' }}"
daemon_reload: true
- name: monky_deployd | bootstrap grant (one-time, from tenancy)
ansible.builtin.copy:
content: "{{ monky_deployd_bootstrap_grant }}\n"
dest: /etc/monky-deployd/bootstrap.jwt
owner: monky-deployd
group: monky-deployd
mode: "0600"
when: monky_deployd_bootstrap_grant | length > 0
no_log: true
notify: monky_deployd tick
- name: monky_deployd | timer
ansible.builtin.systemd:
name: monky-deployd.timer
enabled: "{{ monky_deployd_timer_enabled }}"
state: "{{ 'started' if monky_deployd_timer_enabled else 'stopped' }}"
daemon_reload: true
- name: monky_deployd | flush handlers (first tick inside the grant's hour)
ansible.builtin.meta: flush_handlers
- name: monky_deployd | status
ansible.builtin.command: monky-deployd status
register: monky_deployd_status
changed_when: false
failed_when: false
- name: monky_deployd | show status
ansible.builtin.debug:
msg: "{{ monky_deployd_status.stdout_lines }}"
@@ -0,0 +1,29 @@
# {{ ansible_managed }} — monky-deployd {{ monky_deployd_version }} (role monky_deployd)
env_id: {{ monky_deployd_env_id }}
site: {{ monky_deployd_site }}
transport: {{ monky_deployd_transport }}
identity: {{ monky_deployd_identity }}
tenancy:
service: {{ monky_deployd_tenancy_service }}
host: {{ monky_deployd_tenancy_service }}
port: {{ monky_deployd_tenancy_port }}
scheme: http
proxy_addr: 127.0.0.1:18443
bao:
service: openbao
addr: {{ monky_deployd_bao_addr }}
proxy_addr: 127.0.0.1:18200
ca_bundle: {{ '/etc/monky-deployd/openbao-ca.pem' if monky_deployd_bao_ca_pem | length > 0 else 'none' }}
mount: {{ monky_deployd_bao_auth_mount }}
role: {{ monky_deployd_bao_role }}
kv_mount: {{ monky_deployd_bao_kv_mount }}
state_dir: /var/lib/monky-deployd
deploy_dir: /var/lib/monky-deployd/{{ monky_deployd_env_id }}
bootstrap_path: /etc/monky-deployd/bootstrap.jwt
interval_s: {{ monky_deployd_interval_s }}
healthy_timeout_s: {{ monky_deployd_healthy_timeout_s }}
disk:
factor: {{ monky_deployd_disk_factor }}
headroom_bytes: {{ monky_deployd_disk_headroom_bytes }}
volumes_on_absent: {{ monky_deployd_volumes_on_absent }}
laptop_mode: {{ 'true' if monky_deployd_laptop_mode else 'false' }}
+46
View File
@@ -0,0 +1,46 @@
# /etc/monky-deployd/config.yaml — monky-deployd v0.1.0 (MONKY-ADR-0028 §D)
# Written by packaging/install.sh (or the ansible role monky_deployd). YAML *subset*: maps, scalars,
# simple lists, comments. Keys not listed here are a config error.
env_id: env-qa-02 # env-<tier>-<nn> (or a grandfathered legacy id); MUST match the token's env
site: cbs # cbs | pdx (lowercase DC code)
transport: sdk # sdk (OpenZiti Python SDK, default) | proxy (monky-deployd-proxy.service) | system (tunneler `run` mode / plain DNS)
identity: /opt/openziti/etc/identities/monky-host.env-qa-02.json # the box's host identity (read via ACL)
tenancy:
service: monky.tenancy.deploy # ziti service bound by the tenancy sidecar -> 127.0.0.1:8081 (agent entrypoint)
host: monky.tenancy.deploy # intercept host (sdk/system); defaults to `service`
port: 8081
scheme: http # plain HTTP inside the mesh; the mesh is the transport security
proxy_addr: 127.0.0.1:18443 # transport: proxy
timeout_s: 30
bao:
service: openbao # ziti service (#openbao-client dial), terminates on openbao-active
addr: https://bao.cbs.tikali.net:8200 # intercept name — NOT public DNS; TLS validated against ca_bundle
proxy_addr: 127.0.0.1:18200 # transport: proxy (SNI + cert check still use bao.cbs.tikali.net)
ca_bundle: /etc/monky-deployd/openbao-ca.pem # the openbao-ca certificate (cert-manager CA, not public); `none` = system store
mount: jwt-tenancy # AUTH mount: POST /v1/auth/jwt-tenancy/login {"role": "see-env", "jwt": <deploy grant>}
role: see-env
kv_mount: monky # KV-v2 mount; the agent may read monky/data/<env_id>/see/* only
token_max_ttl_s: 2592000 # 30 d — re-lease `release_before_s` before this age
renew_below_s: 43200 # renew-self when the remaining TTL drops under 12 h
release_before_s: 172800 # re-lease 2 d before max TTL
timeout_s: 30
state_dir: /var/lib/monky-deployd # bao.token (0600), state.json, lock
deploy_dir: /var/lib/monky-deployd/env-qa-02 # releases/<sha>/ + current -> the compose project directory
bootstrap_path: /etc/monky-deployd/bootstrap.jwt # the kit's one-time deploy grant; consumed and deleted on first login
interval_s: 60 # loop-mode sleep (the systemd timer is the normal driver)
healthy_timeout_s: 300 # wait for `compose ps` to be healthy after `up`
disk:
factor: 1.5 # refuse when docker data-root free < need x factor + headroom (the env-dev-09 lesson)
headroom_bytes: 2147483648 # 2 GiB
volumes_on_absent: keep # keep | purge — what `down` does with data volumes (never purge on prod)
laptop_mode: false # true: offline exits 0 quietly; run without the timer (`monky-deployd run`)
# prod: false # override tier detection (env-prod-* / prod-cedar are prod)
# compose_project: monky-env-qa-02 # docker compose project name
# docker_bin: docker
# log_level: INFO
+109
View File
@@ -0,0 +1,109 @@
<!-- xlate:verbatim-fences -->
# Operating monky-deployd
## Units
| unit | what |
|---|---|
| `monky-deployd.timer` | fires `monky-deployd.service` 60 s after the previous tick finished (+ ≤10 s jitter); `OnBootSec=90s` |
| `monky-deployd.service` | `Type=oneshot`, `monky-deployd run --once` as user `monky-deployd` (+ `docker` group); `SuccessExitStatus=75`; hardened (`NoNewPrivileges`, `ProtectSystem=strict`, `ReadWritePaths=/var/lib/monky-deployd /etc/monky-deployd /run/docker.sock`, `UMask=0077`, no capabilities) |
| `monky-deployd-proxy.service` | only with `transport: proxy`: `ziti tunnel proxy -i <identity> monky.tenancy.deploy:18443 openbao:18200` as user `ziti`; `EnvironmentFile=/etc/monky-deployd/proxy.env` |
| `ziti-edge-tunnel.service` | the host identity's tunneler in **`run-host`** mode (drop-in `run-host.conf` written by `install.sh`) |
```sh
systemctl status monky-deployd.timer monky-deployd.service
systemctl list-timers monky-deployd.timer
systemctl start monky-deployd.service # tick now (blocks until done)
journalctl -u monky-deployd -f # priorities are real (INFO/WARNING/ERROR)
monky-deployd status # exit 0 = token present and in sync
monky-deployd status --json | jq .
```
Files: `/etc/monky-deployd/config.yaml` (0640 root:monky-deployd), `/etc/monky-deployd/openbao-ca.pem`,
`/etc/monky-deployd/bootstrap.jwt` (only until the first login), `/var/lib/monky-deployd/{bao.token,state.json,lock}`,
`/var/lib/monky-deployd/<env>/releases/<sha>/` + `current` (the compose project dir, `.env` 0600),
`/opt/openziti/etc/identities/monky-host.<env>.json` (ziti:ziti 0600 + ACL `u:monky-deployd:r`).
## Reading the journal
| line | meaning |
|---|---|
| `checkin: action=none desired=… applied=…` then `healthy; heartbeat reported` | converged |
| `checkin: action=apply …``read 3 secret(s): GEMINI_API_KEY, …``promoted release …``applied …` | a deploy |
| `refused: ENV_INCOMPLETE: unresolved: X` | the bundle needs a variable no manifest entry supplies — fix the descriptor / set the secret in the console; nothing was started |
| `refused: DISK_INSUFFICIENT: docker data-root has N MiB free, bundle needs M MiB` | free space (the env-dev-09 lesson): grow the data-root disk or prune |
| `refused: PRIVILEGED_REFUSED` / `ROLLBACK_REFUSED` | the bundle needs `allow_privileged` / `allow_rollback` in its `agent` profile |
| `temporary network failure` (exit 75) | mesh/tenancy unreachable — check `ziti-edge-tunnel`, the identity's terminators, `monky.tenancy.deploy` health |
| `AGENT_UNAUTHENTICATED: bearer refused` (exit 1) | the grant was superseded (kit re-revealed / retire) or the token revoked → re-run the install kit |
| `AGENT_ENV_MISMATCH` (exit 78) | the token belongs to another env than `config.yaml` — fix the config or re-issue the identity; the timer keeps firing but every tick exits 78 immediately (no storm) |
| `failed: docker compose pull failed (rc=1)` | registry/pull problem; compose output is in the report's tail and in the journal |
Values never appear in the journal (`[REDACTED]` for token shapes and every value the agent has read).
## Secrets on the box
The only credential is `/var/lib/monky-deployd/bao.token` (0600, user `monky-deployd`). `.env`
files under `releases/<sha>/` hold the rendered values (0600, same user, `docker compose` reads
them). Rotating a secret in the console changes the bundle sha → the next tick re-reads and
re-applies. To force a re-read now: `systemctl start monky-deployd.service`.
Lost or revoked token: `monky-deployd status` shows `token ABSENT`; reveal the kit again (admin,
`GET /v1/backends/{id}/agent/install`), paste its bootstrap grant to `/etc/monky-deployd/bootstrap.jwt`
(0600 monky-deployd) or re-run `install.sh` (enrolment is skipped when the identity exists), then
`monky-deployd bootstrap` or wait a tick.
## Retire and volumes policy
Retire is driven by tenancy (`POST /v1/backends/{id}/retire {confirm, force, purge_volumes}`):
the next check-in returns `action: down` and the agent runs `docker compose down --remove-orphans`.
Data volumes:
- `-v` (remove volumes) only when tenancy sent `purge_volumes: true` **or** the box's
`volumes_on_absent: purge` — and **never on a prod env** (the request is logged and ignored).
- default `volumes_on_absent: keep`: volumes stay for a manual `docker volume rm` later.
After `down` the agent reports `down`, clears `applied_sha`, removes `current` and keeps
`releases/` (no secrets outside `.env`, which you can `shred`). Tenancy then destroys the Bao
paths, revokes the token accessor and deletes the host identity → the following ticks fail with
`AGENT_UNAUTHENTICATED`; `systemctl disable --now monky-deployd.timer` and `apt remove monky-deployd`
(`apt purge` also removes `/var/lib/monky-deployd` and `/etc/monky-deployd`).
## Laptop mode
`laptop_mode: true` (`install.sh --laptop`): being offline is normal — a tick that cannot reach the
mesh logs one INFO line and exits 0. Run without the system timer:
```sh
monky-deployd run # loop, interval_s + jitter, SIGTERM stops; or a user timer with `run --once`
```
Liveness grace for laptops is 24 h on the tenancy side (P0); `offline` only hides the backend from
the picker, nothing is retired automatically.
## Housekeeping
`monky-deployd run --once --prune` removes every `releases/<sha>` except `current` and runs
`docker image prune -f` after a successful apply. The state dir and the token survive `apt remove`;
`apt purge` deletes them.
## Publishing a release to Gitea by hand
When `release:gitea` is manual (no `GITEA_TOKEN` in CI) — from jump1 with the Gitea API token:
```sh
T=$(tr -d '\n' < ~/.gitlab_tokens/gitea-token); G=http://172.16.8.1:3000/api/v1/repos/mdella/monky-deployd
curl -s -X POST -H "Authorization: token $T" $G/mirror-sync # pull the tag
RID=$(curl -s -X POST -H "Authorization: token $T" -H 'Content-Type: application/json' $G/releases \
-d '{"tag_name":"v0.1.0","name":"monky-deployd v0.1.0","body":"See CHANGELOG.md"}' | jq -r .id)
for f in monky-deployd_0.1.0_amd64.deb monky-deployd_0.1.0_amd64.deb.sha256 install.sh; do
curl -s -X POST -H "Authorization: token $T" -F "attachment=@$f" "$G/releases/$RID/assets?name=$f"; done
```
The assets are then at `https://gitea.cbs.tikali.net/mdella/monky-deployd/releases/download/v0.1.0/<file>`,
which is what `install.sh` fetches (GitLab artifacts from the tag pipeline's `package` job).
## Ansible (osg1-07)
`ansible/roles/monky_deployd/` is the role skeleton to copy into osg1-07: `.deb` from the Gitea
release (sha256-verified), config template, bootstrap grant from a vault var, ACL on the identity,
timer, one tick. Rolled to env-dev-06..09 after the env-qa-02 pilot; env-dev-01 last.
+148
View File
@@ -0,0 +1,148 @@
<!-- xlate:verbatim-fences -->
# monky-deployd protocol
The agent side of the backend lifecycle agent protocol — [MONKY-ADR-0028](https://scm.tikali.ai/tikali/applications/monky/monky-design-docs/-/blob/develop/adr/0028-backend-lifecycle-ownership-and-execution.md)
§2/§3 (amendment 2026-09-05) and [design doc 24](https://scm.tikali.ai/tikali/applications/monky/monky-design-docs/-/blob/develop/docs/24-backend-lifecycle.md)
§2.1a, §3.3, §4. The tenancy side is monky-tenancy `app/agent_main.py` / `app/api/agent.py` /
`app/schemas_backends.py`; its shapes are the contract, this page is how the agent uses them.
## Where and how
- **Where:** tenancy's **agent entrypoint** — a second container (`app.agent_main`,
`127.0.0.1:8081`) bound to the ziti service **`monky.tenancy.deploy`** by a `ziti-edge-tunnel
run-host` sidecar. Unreachable from the public ingress. Only host identities carrying
`#monky-deploy-agent` may dial (the broker adds the attr when the identity is created at kit reveal).
- **Transport:** plain HTTP inside the mesh (`transport: sdk` dials by service name;
`proxy``127.0.0.1:18443`). OpenBao is the existing `openbao` ziti service (`#openbao-client`),
dialled as `https://bao.cbs.tikali.net:8200` (an intercept name), TLS validated against the
`openbao-ca` certificate (`bao.ca_bundle`).
- **Auth:** `Authorization: Bearer <the agent's OpenBao token>`. Tenancy verifies it with
`auth/token/lookup` (cached 60 s), pins the request's `env_id` to the token's `meta.env_id`
(`403 AGENT_ENV_MISMATCH` → exit 78, never retried), and compares `meta.grant_jti` against
`backend_leases` (`401 AGENT_UNAUTHENTICATED` when the grant was superseded or revoked).
- **Rate buckets per env:** checkin 6/min, bundle 10/min, report 30/min, lease **5/h**
(`429 LEASE_RATE_LIMITED`, `Retry-After`) → the agent treats 429 as temporary (exit 75).
- **Errors** are `{code, detail}` JSON; the agent maps them to `TenancyError` subclasses.
## Bootstrap → lease → token (the grant flow)
```
kit reveal (admin, once) tenancy signs a bootstrap DEPLOY GRANT: ES256 JWT
iss <tenancy issuer>, aud openbao-see-env, sub agent:<env_id>,
env_id, kind deploy-grant, jti (single-use, recorded), exp now+1h
install.sh stages it at /etc/monky-deployd/bootstrap.jwt (0600)
first tick POST https://bao…/v1/auth/jwt-tenancy/login {"role":"see-env","jwt":<grant>}
→ auth.client_token (ttl 24 h, max 30 d, policy see-env,
metadata {env_id, grant_jti} from the mount's claim_mappings)
token → /var/lib/monky-deployd/bao.token (0600); grant deleted
every tick Bearer <token> → checkin / bundle / lease / report
apply POST /v1/agent/lease → {login_jwt,…} → jwt-tenancy login → NEW token
(becomes the bearer; the old one is revoke-self'd) → KV reads
upkeep (finally) lookup-self; ttl < 12 h → renew-self; age > 30 d 2 d → re-lease
```
OpenBao side (Terraform, `tikali/services/security/openbao`): mount `jwt-tenancy`
(`jwks_url` = tenancy's `GET /.well-known/agent-jwks.json`, in-cluster), role `see-env`
(`role_type=jwt`, `user_claim=env_id` ⇒ one alias/entity **per env**, `bound_audiences=["openbao-see-env"]`,
`bound_claims={"kind":"deploy-grant"}`, `claim_mappings={"env_id":"env_id","jti":"grant_jti"}`,
`token_ttl=86400`, `token_max_ttl=2592000`, `token_no_default_policy=true`), policy `see-env` =
`monky/data/{{identity.entity.aliases.<accessor>.metadata.env_id}}/see/*` read + metadata list +
`auth/token/{renew-self,lookup-self}`. **Gate 1 v2 passed 2026-09-05 07:40Z** (distinct entity per env).
**Second-reveal semantics.** A kit re-reveal (or a retire) supersedes every earlier grant of the
backend and revokes the token accessors tenancy knows. The old kit's grant fails at login
(unknown/used `jti`), and a token already minted from it is refused at the next call with
`401 AGENT_UNAUTHENTICATED` (its `meta.grant_jti` is superseded) — **not** `AGENT_ENV_MISMATCH`.
The agent then deletes its token; if a fresh `bootstrap.jwt` is on disk it bootstraps again in the
same tick, otherwise it exits 1 and says "re-run the install kit".
## The four calls
### `POST /v1/agent/checkin`
```json
{"env_id": "env-qa-02", "agent_version": "0.1.0", "applied_sha": "3f9c…",
"host": {"hostname": "env-qa-02", "os": "ubuntu-26.04", "kernel": "7.0.0-30-generic",
"docker": "28.3.0", "compose": "2.32.0", "free_bytes": 61234567890,
"agent_version": "0.1.0", "transport": "sdk", "site": "cbs", "laptop_mode": false},
"containers": [{"name": "see-backend", "state": "running", "health": "healthy"}]}
```
```json
{"env_id": "env-qa-02", "desired_sha": "7a10…", "action": "apply", "purge_volumes": false,
"bundle_url": "/v1/agent/bundle/env-qa-02/7a10…", "checkin_interval_s": 60,
"vault": {"addr": "https://bao.cbs.tikali.net:8200", "mount": "monky", "prefix": "env-qa-02/see"}}
```
`action`: `apply` (desired ≠ applied), `none` (converged → heartbeat), `down` (retire; `purge_volumes`
is only meaningful here). `vault.mount` is the **KV** mount; the agent adopts it if it differs from
config. `vault.addr` is informational — the transport decides how OpenBao is reached.
### `GET /v1/agent/bundle/{env_id}/{sha}`
`application/x-tar`, `Cache-Control: no-store`, `X-Bundle-Sha` (tenancy) / `X-Bundle-Sha256` (doc 24;
both accepted). Members: `docker-compose.yml` (or `compose.yaml`), `.env.template`,
`secrets.manifest.json`, `bundle.json`, optionally `.env.example`. **No secret material** — the agent
refuses a manifest entry carrying a `value`. The agent recomputes
```
sha256( for name in sorted(files): name + "\0" + content + "\0" )
```
and refuses (`BUNDLE_SHA_MISMATCH`) unless it equals `desired_sha`. `404 BUNDLE_NOT_FOUND` when
the sha is not published.
`secrets.manifest.json` (paths + versions, never values):
```json
{"vault_mode": "openbao",
"entries": [{"var": "GEMINI_API_KEY", "path": "monky/env-qa-02/see/gemini_api_key", "kind": "supplied", "version": 2},
{"var": "POSTGRES_PASSWORD", "path": "monky/env-qa-02/see/pg_password", "kind": "generated", "version": 1},
{"var": "SEE_ADMIN_TOKEN", "path": "monky/env-qa-02/see/admin_token", "kind": "generated", "version": 1}]}
```
Path forms `monky/<env>/see/<name>` (tenancy) and `monky/data/<env>/see/<name>` (doc 24) both map
to `GET /v1/monky/data/<env>/see/<name>?version=N`; the `<env>` segment **must** equal the agent's
`env_id`. `version: null` = latest. The secret document is `{"value": "…"}`.
`bundle.json` (monky-deploy `render_bundle_json`): `env_id, tier, site, bundle, target, target_class,
address, pull, renderer, images_policy, secrets_provider, agent{…}, files[]`. The agent honours
`allow_privileged`, `allow_rollback`, `disk_need_bytes` (top level or under `agent`), refuses a
`renderer` other than `compose`, and treats `tier == "prod"` as prod.
### `POST /v1/agent/lease`
```json
{"env_id": "env-qa-02", "reason": "apply"} // reason: apply | renew
```
```json
{"env_id": "env-qa-02", "login_jwt": "eyJ…", "ttl_s": 3600, "mount": "jwt-tenancy", "role": "see-env",
"vault": {"addr": "https://bao.cbs.tikali.net:8200", "mount": "monky"}}
```
Then `POST /v1/auth/{mount}/login {"role": "{role}", "jwt": "{login_jwt}"}` on OpenBao. A body with
`wrapping_token` / `role_id` (the pre-Gate-1 AppRole lease) is refused with `LEASE_SHAPE` → report
`failed`. `429 LEASE_RATE_LIMITED` → exit 75.
### `POST /v1/agent/report`
```json
{"env_id": "env-qa-02", "sha": "7a10…", "result": "applied",
"log_tail": "INFO checkin: action=apply …\nINFO applied 7a10c0ffee11 (2 container(s) healthy)",
"containers": [{"name": "see-backend", "state": "running", "health": "healthy"}],
"detail": "ENV_INCOMPLETE: unresolved: SEE_ADMIN_TOKEN"} // detail only on failed
```
`result`: `applied` (sets `applied_sha`, health ok — also the heartbeat on `none`), `failed`
(health degraded; `detail` = code + names), `down` (clears `applied_sha`). `log_tail` ≤ 16 KiB, the
first 2 KiB land in the audit log — it has been through the redactor.
## Divergences (2026-09-05)
- **monky-tenancy `main` (MR !15) still implements the AppRole lease and install kit**
(`AgentLeaseOut{wrapping_token, role_id}`, `bootstrap.wrap`, `bao.approle` in the kit's config). The
binding design is the plan's Gate 1 RESULT / ADR-0028 amendment: `{login_jwt, ttl_s, mount, role}`
and `POST /v1/auth/jwt-tenancy/login`. This agent implements the latter; against an un-migrated
tenancy it reports `failed` with `LEASE_SHAPE` and refuses `bao.approle` in its config. The tenancy
follow-up (deploy-grant signer, JWKS, `lease` shape, kit → `bootstrap.jwt`) is tracked on
monky-tenancy.
- The kit's generated config uses `tenancy.base_url: http://monky.tenancy.deploy:8081` — accepted as
an alias for `tenancy.{scheme,host,port}`.
- `report` gains an optional `detail` (doc 24 §3.3); tenancy's `AgentReport` ignores unknown fields
today — if `strict` bodies land, `detail` folds into `log_tail`.
- Bundle sha header: tenancy sends `X-Bundle-Sha`, doc 24 says `X-Bundle-Sha256`; the agent reads
either and trusts only its own computation.
+7
View File
@@ -0,0 +1,7 @@
"""monky-deployd — the on-box pull agent for Monky backends (MONKY-ADR-0028).
Dials monky-tenancy over the mesh with the box's host identity, fetches the rendered bundle,
leases a deploy grant, logs in to OpenBao, reads its own secrets, runs `docker compose`,
reports. Stdlib only; the optional `openziti` SDK is the `sdk` transport."""
__version__ = "0.1.0"
+3
View File
@@ -0,0 +1,3 @@
from monky_deployd.cli import main
raise SystemExit(main())
+548
View File
@@ -0,0 +1,548 @@
"""One reconcile tick — the loop of MONKY-ADR-0028 §D, with Reconciliation v2 vocabulary:
flock -> load state -> checkin (bearer = Bao token; bootstrap it from the kit's grant if
absent) -> action:
down -> compose down --remove-orphans [-v] -> report down
none -> compose ps; healthy? -> report applied (heartbeat)
apply -> bundle (sha verified) -> refusal checks -> lease -> jwt-tenancy login
-> KV reads -> .env 0600 -> promote -> pull -> up -d --remove-orphans
-> wait healthy -> report applied | failed
finally: token upkeep (renew-self / re-lease before max TTL), scrub, save state.
Exit codes: 0 ok (or offline in laptop mode) · 75 temporary network failure · 78 AGENT_ENV_MISMATCH
(never retried) · 1 refusal / failure."""
from __future__ import annotations
import logging
import os
import platform
import shutil
import socket
import time
from pathlib import Path
from monky_deployd import __version__
from monky_deployd import bundle as bundlemod
from monky_deployd import state as statemod
from monky_deployd.bao import BaoClient, BaoError, BaoToken, ManifestPathError, kv_data_path
from monky_deployd.bundle import Bundle, BundleError
from monky_deployd.compose import Compose, ComposeError, Container, Docker
from monky_deployd.config import Config
from monky_deployd.redact import REDACTOR
from monky_deployd.state import Lock, State
from monky_deployd.tenancy import (
Checkin,
EnvMismatch,
LeaseShapeUnsupported,
RateLimited,
TenancyClient,
TenancyError,
Unauthenticated,
)
from monky_deployd.transport import HttpClient, TransportError, build
log = logging.getLogger("monky-deployd")
EX_OK, EX_FAIL, EX_TEMPFAIL, EX_ENV_MISMATCH = 0, 1, 75, 78
LEASE_MIN_GAP_S = 15 * 60 # never re-lease more often than this (tenancy allows 5/h)
class Refusal(Exception):
"""The agent refuses to apply; reported to tenancy as `failed` with the code + names only."""
def __init__(self, code: str, detail: str):
super().__init__(f"{code}: {detail}")
self.code, self.detail = code, detail
class NoCredentials(Exception):
pass
class TailHandler(logging.Handler):
"""Keeps the (redacted) last 16 KiB of this tick's log for the report's `log_tail`."""
def __init__(self, limit: int = 16 * 1024):
super().__init__(logging.INFO)
self.limit = limit
self.lines: list[str] = []
self.size = 0
self.addFilter(REDACTOR)
self.setFormatter(logging.Formatter("%(levelname)s %(message)s"))
def emit(self, record: logging.LogRecord) -> None:
line = self.format(record)
self.lines.append(line)
self.size += len(line) + 1
while self.size > self.limit and self.lines:
self.size -= len(self.lines.pop(0)) + 1
def text(self) -> str:
return REDACTOR.scrub("\n".join(self.lines))
class Agent:
def __init__(self, cfg: Config, *, prune: bool = False, docker: Docker | None = None):
self.cfg = cfg
self.prune = prune
self.docker = docker or Docker(cfg.docker_bin)
self.transport = build(cfg)
self.tenancy_http = HttpClient(
self.transport, cfg.tenancy.scheme, cfg.tenancy.host, cfg.tenancy.port, timeout=cfg.tenancy.timeout_s
)
scheme, host, port = cfg.bao_url
self.bao_http = HttpClient(
self.transport, scheme, host, port, ca_bundle=cfg.bao.ca_bundle, timeout=cfg.bao.timeout_s
)
self.bao = BaoClient(self.bao_http, auth_mount=cfg.bao.mount, role=cfg.bao.role, kv_mount=cfg.bao.kv_mount)
self.state: State = State(env_id=cfg.env_id)
self.token: str | None = None
self.tenancy: TenancyClient | None = None
self.tail = TailHandler()
self._staging: list[Path] = []
self._values: dict[str, str] = {}
self._releases_this_tick = 0
# -- entry -------------------------------------------------------------------------------------
def run_once(self) -> int:
lock = Lock(self.cfg.lock_path)
if not lock.acquire():
log.info("another monky-deployd tick holds %s; skipping", self.cfg.lock_path)
return EX_OK
lg = logging.getLogger("monky-deployd")
if lg.getEffectiveLevel() > logging.INFO:
lg.setLevel(logging.INFO) # the tail must see INFO even when the root is quieter
lg.addHandler(self.tail)
try:
self.state = statemod.load(self.cfg.state_path, self.cfg.env_id)
self.state.agent_version = __version__
return self._guarded_tick()
finally:
try:
self._maintain_token()
except Exception as exc: # never let upkeep mask the tick's own result
log.warning("token upkeep skipped: %s", REDACTOR.scrub(str(exc)))
self._scrub()
try:
statemod.save(self.cfg.state_path, self.state)
except OSError as exc:
log.error("cannot save state: %s", exc)
lg.removeHandler(self.tail)
lock.release()
def _guarded_tick(self) -> int:
try:
return self._tick()
except TransportError as exc:
self.state.last_error = f"network: {exc}"
if self.cfg.laptop_mode:
log.info("offline (%s); laptop mode, nothing to do", exc)
return EX_OK
log.warning("temporary network failure: %s", exc)
return EX_TEMPFAIL
except EnvMismatch as exc:
self.state.last_error = f"AGENT_ENV_MISMATCH: {exc.detail}"
log.error(
"AGENT_ENV_MISMATCH: this box's token is pinned to another environment (%s); "
"config says %s. Not retrying — fix the config or re-run the install kit.",
exc.detail,
self.cfg.env_id,
)
return EX_ENV_MISMATCH
except Unauthenticated as exc:
self.state.last_error = f"AGENT_UNAUTHENTICATED: {exc.detail}"
log.error(
"AGENT_UNAUTHENTICATED: bearer refused (%s). The deploy grant was superseded or the "
"token revoked; re-run the install kit (a fresh bootstrap grant).",
exc.detail,
)
return EX_FAIL
except NoCredentials as exc:
self.state.last_error = str(exc)
log.error("%s", exc)
return EX_FAIL
except RateLimited as exc:
self.state.last_error = f"{exc.code}: retry after {exc.retry_after}s"
log.warning("%s (retry-after %ss)", exc, exc.retry_after)
return EX_TEMPFAIL
except (Refusal, BundleError, ManifestPathError, LeaseShapeUnsupported) as exc:
code = getattr(exc, "code", exc.__class__.__name__)
detail = getattr(exc, "detail", str(exc))
self.state.last_error = f"{code}: {detail}"
self.state.last_result = "failed"
log.error("refused: %s: %s", code, detail)
self._report("failed", self.state.desired_sha, detail=f"{code}: {detail}")
return EX_FAIL
except (TenancyError, BaoError, ComposeError) as exc:
self.state.last_error = str(exc)
self.state.last_result = "failed"
log.error("failed: %s", exc)
self._report("failed", self.state.desired_sha, detail=str(exc))
return EX_FAIL
# -- the tick ------------------------------------------------------------------------------------
def _tick(self) -> int:
cfg = self.cfg
log.info("tick env=%s site=%s transport=%s", cfg.env_id, cfg.site, self.transport.describe())
self._ensure_token()
containers = self._compose().ps() if self.docker.available() else []
ci = self._checkin(containers)
self.state.last_checkin_at = time.time()
self.state.last_action = ci.action
self.state.desired_sha = ci.desired_sha
if ci.vault.mount and ci.vault.mount != self.bao.kv_mount:
log.info("KV mount from tenancy: %s", ci.vault.mount)
self.bao.kv_mount = ci.vault.mount
log.info(
"checkin: action=%s desired=%s applied=%s",
ci.action,
_short(ci.desired_sha),
_short(self.state.applied_sha),
)
if ci.action == "down":
return self._do_down(ci)
if ci.action == "none":
return self._do_none(containers)
return self._do_apply(ci)
def _checkin(self, containers: list[Container]) -> Checkin:
assert self.tenancy is not None
kwargs = dict(
agent_version=__version__,
applied_sha=self.state.applied_sha,
host=self._host_facts(),
containers=[c.as_dict() for c in containers] or None,
)
try:
return self.tenancy.checkin(**kwargs)
except Unauthenticated as exc:
# a superseded grant / revoked token: drop it; a bootstrap grant on disk rescues us once
log.warning("checkin refused (%s); discarding the stored token", exc.detail)
statemod.delete(self.cfg.token_path)
self.state.token = statemod.TokenMeta()
self.token = None
if not Path(self.cfg.bootstrap_path).exists():
raise
self._ensure_token()
return self.tenancy.checkin(**kwargs)
# -- actions ---------------------------------------------------------------------------------------
def _do_none(self, containers: list[Container]) -> int:
if not self.state.applied_sha:
log.info("nothing desired, nothing applied; idle")
return EX_OK
healthy = bool(containers) and all(c.ok for c in containers)
if healthy:
self._report("applied", self.state.applied_sha, containers=containers)
log.info("healthy; heartbeat reported for %s", _short(self.state.applied_sha))
return EX_OK
bad = ", ".join(f"{c.name}={c.state}/{c.health or '-'}" for c in containers) or "no containers"
log.warning("unhealthy: %s", bad)
self.state.last_error = f"unhealthy: {bad}"
self._report("failed", self.state.applied_sha, containers=containers, detail=f"unhealthy: {bad}")
return EX_FAIL
def _do_down(self, ci: Checkin) -> int:
purge = ci.purge_volumes or self.cfg.volumes_on_absent == "purge"
if purge and self.cfg.is_prod:
log.warning("purge_volumes requested on a prod backend: REFUSED, volumes kept")
purge = False
compose = self._compose()
log.info("desired action down: compose down --remove-orphans%s", " -v" if purge else "")
if self.docker.available():
compose.down(purge_volumes=purge)
current = Path(self.cfg.deploy_dir) / "current"
if current.is_symlink() or current.exists():
current.unlink()
self.state.applied_sha = None
self.state.last_result = "down"
self._report("down", None)
return EX_OK
def _do_apply(self, ci: Checkin) -> int:
assert self.tenancy is not None
cfg = self.cfg
if not ci.desired_sha or not ci.bundle_url:
raise Refusal("BAD_CHECKIN", "action apply without desired_sha/bundle_url")
if not self.docker.available():
raise Refusal("DOCKER_MISSING", f"{cfg.docker_bin} is not on PATH")
data, hdr_sha = self.tenancy.bundle(ci.bundle_url)
b = bundlemod.parse(data)
if b.sha != ci.desired_sha:
raise Refusal("BUNDLE_SHA_MISMATCH", f"computed {b.sha[:12]} != desired {ci.desired_sha[:12]}")
if hdr_sha and hdr_sha != b.sha:
log.warning("bundle header sha %s disagrees with content %s", _short(hdr_sha), _short(b.sha))
self._refusal_checks(b)
# secrets: lease -> login -> reads (values never logged; names only)
entries = b.manifest.get("entries", [])
if entries:
token = self._lease_login("apply")
for e in entries:
self._values[e["var"]] = self.bao.kv_read(token, e["path"], cfg.env_id, e.get("version"))
log.info("read %d secret(s): %s", len(entries), ", ".join(sorted(self._values)))
env_text = bundlemod.render_env(b.env_template, self._values)
leftover = bundlemod.referenced_vars(env_text)
if leftover:
raise Refusal("ENV_INCOMPLETE", "unfilled after render: " + ", ".join(sorted(leftover)))
release = self._promote(self._stage(b, env_text))
compose = self._compose()
log.info("compose pull")
compose.pull()
log.info("compose up -d --remove-orphans")
compose.up()
ok, containers = compose.wait_healthy(cfg.healthy_timeout_s)
if not ok:
bad = ", ".join(f"{c.name}={c.state}/{c.health or '-'}" for c in containers) or "no containers"
tail = compose.logs_tail(60)
self.state.last_result = "failed"
self.state.last_error = f"unhealthy after up: {bad}"
log.error("not healthy within %ss: %s", cfg.healthy_timeout_s, bad)
self._report("failed", b.sha, containers=containers, detail=f"unhealthy: {bad}", extra_tail=tail)
return EX_FAIL
self.state.remember_applied(b.sha)
self.state.last_result = "applied"
self.state.last_error = None
self._report("applied", b.sha, containers=containers)
log.info("applied %s (%d container(s) healthy)", _short(b.sha), len(containers))
if self.prune:
self._prune(release)
return EX_OK
# -- checks ----------------------------------------------------------------------------------------
def _refusal_checks(self, b: Bundle) -> None:
cfg = self.cfg
if b.env_id and b.env_id != cfg.env_id:
raise Refusal("BUNDLE_ENV_MISMATCH", f"bundle is for {b.env_id}, this box is {cfg.env_id}")
if b.meta.get("renderer") not in (None, "compose"):
raise Refusal("BUNDLE_RENDERER", f"renderer {b.meta.get('renderer')!r} is not compose")
missing = bundlemod.unresolved_vars(b, bundlemod.manifest_vars(b))
if missing:
raise Refusal("ENV_INCOMPLETE", "unresolved: " + ", ".join(missing))
for e in b.manifest.get("entries", []):
kv_data_path(self.bao.kv_mount, e["path"], cfg.env_id) # raises ManifestPathError
findings = bundlemod.privileged_findings(b)
if findings and not b.flag("allow_privileged"):
raise Refusal("PRIVILEGED_REFUSED", ", ".join(findings) + " (bundle.json allow_privileged is not set)")
if self.state.is_rollback(b.sha) and not b.flag("allow_rollback"):
raise Refusal("ROLLBACK_REFUSED", f"{b.sha[:12]} was applied before; allow_rollback is not set")
need = b.disk_need_bytes
if need:
free = self.docker.free_bytes()
required = int(need * cfg.disk.factor + cfg.disk.headroom_bytes)
if free is not None and free < required:
raise Refusal(
"DISK_INSUFFICIENT",
f"docker data-root has {free // 2**20} MiB free, bundle needs {required // 2**20} MiB "
f"({need // 2**20} MiB x {cfg.disk.factor} + {cfg.disk.headroom_bytes // 2**20} MiB headroom)",
)
# -- credentials -------------------------------------------------------------------------------------
def _ensure_token(self) -> str:
if self.token:
return self.token
tok = statemod.read_token(self.cfg.token_path)
if tok:
REDACTOR.add(tok)
self._adopt(tok, None)
return tok
grant = self._read_bootstrap()
if not grant:
raise NoCredentials(
f"no {self.cfg.token_path} and no {self.cfg.bootstrap_path}: run the install kit "
"(GET /v1/backends/{id}/agent/install) to get a fresh bootstrap grant"
)
log.info("bootstrapping: jwt-tenancy login with the install kit's deploy grant")
bt = self.bao.login(grant)
self._adopt(bt.client_token, bt, source="bootstrap")
if statemod.delete(Path(self.cfg.bootstrap_path)):
log.info("bootstrap grant consumed and deleted")
return bt.client_token
def _read_bootstrap(self) -> str | None:
try:
grant = Path(self.cfg.bootstrap_path).read_text().strip()
except FileNotFoundError:
return None
if grant:
REDACTOR.add(grant)
return grant or None
def _adopt(self, token: str, bt: BaoToken | None, *, source: str | None = None) -> None:
"""Make `token` the bearer: persist 0600, point the tenancy client at it, record meta."""
old = self.token
self.token = token
if bt is not None or old != token:
statemod.write_token(self.cfg.token_path, token)
if bt is not None:
self.state.token = statemod.TokenMeta(
accessor=bt.accessor,
issued_at=time.time(),
ttl_s=bt.ttl_s,
renewable=bt.renewable,
grant_jti=bt.grant_jti,
source=source,
)
if bt.meta.get("env_id") and bt.meta["env_id"] != self.cfg.env_id:
raise EnvMismatch(403, "AGENT_ENV_MISMATCH", f"token env_id={bt.meta['env_id']}")
if self.tenancy is None:
self.tenancy = TenancyClient(self.tenancy_http, self.cfg.env_id, token)
else:
self.tenancy.token = token
if old and old != token:
self.bao.revoke_self(old)
def _lease_login(self, reason: str) -> str:
assert self.tenancy is not None
lease = self.tenancy.lease(reason)
self._releases_this_tick += 1
bt = self.bao.login(lease.login_jwt, mount=lease.mount, role=lease.role)
log.info("lease: jwt-tenancy login ok (ttl %ss, accessor %s)", bt.ttl_s, bt.accessor)
self._adopt(bt.client_token, bt, source="lease")
return bt.client_token
def _maintain_token(self) -> None:
"""Renew-self when the TTL runs low; re-lease before max TTL or when renewal is refused."""
if not self.token or self.tenancy is None or self._releases_this_tick:
return
cfg = self.cfg
try:
info = self.bao.lookup_self(self.token)
except BaoError as exc:
if exc.status in (403, 400):
log.warning("stored token is dead (%s); dropping it", exc)
statemod.delete(cfg.token_path)
self.state.token = statemod.TokenMeta()
self.token = None
return
ttl = int(info.get("ttl") or 0)
renewable = bool(info.get("renewable"))
creation = info.get("creation_time")
age = time.time() - float(creation) if creation else (time.time() - (self.state.token.issued_at or time.time()))
max_ttl = int(info.get("explicit_max_ttl") or 0) or cfg.bao.token_max_ttl_s
near_max = age > max_ttl - cfg.bao.release_before_s
last_lease = self.state.token.issued_at or 0
if near_max or (ttl < cfg.bao.renew_below_s and not renewable):
if time.time() - last_lease < LEASE_MIN_GAP_S:
return
log.info("token age %dh of max %dh: re-leasing", age // 3600, max_ttl // 3600)
self._lease_login("renew")
return
if ttl < cfg.bao.renew_below_s and renewable:
new_ttl = self.bao.renew_self(self.token)
log.info("token renewed (ttl %ss -> %ss)", ttl, new_ttl)
self.state.token.ttl_s = new_ttl
if new_ttl < cfg.bao.renew_below_s and time.time() - last_lease >= LEASE_MIN_GAP_S:
log.info("renewal capped by max TTL: re-leasing")
self._lease_login("renew")
# -- files -------------------------------------------------------------------------------------------
def _stage(self, b: Bundle, env_text: str) -> Path:
deploy = Path(self.cfg.deploy_dir)
deploy.mkdir(parents=True, exist_ok=True, mode=0o700)
staging = deploy / f"staging-{b.sha[:12]}-{os.getpid()}"
if staging.exists():
shutil.rmtree(staging)
staging.mkdir(mode=0o700)
self._staging.append(staging)
for name, content in b.files.items():
if name == bundlemod.ENV_TEMPLATE:
continue
target = staging / name
target.parent.mkdir(parents=True, exist_ok=True)
statemod.write_private(target, content.encode())
os.chmod(target, 0o600)
statemod.write_private(staging / ".env", env_text.encode())
(staging / "SHA256").write_text(b.sha + "\n")
return staging
def _promote(self, staging: Path) -> Path:
deploy = Path(self.cfg.deploy_dir)
releases = deploy / "releases"
releases.mkdir(exist_ok=True, mode=0o700)
sha = (staging / "SHA256").read_text().strip()
release = releases / sha
if release.exists():
shutil.rmtree(release)
os.replace(staging, release)
self._staging.remove(staging)
tmp_link = deploy / ".current.tmp"
if tmp_link.is_symlink() or tmp_link.exists():
tmp_link.unlink()
os.symlink(release, tmp_link)
os.replace(tmp_link, deploy / "current")
log.info("promoted release %s", _short(sha))
return release
def _prune(self, keep: Path) -> None:
releases = Path(self.cfg.deploy_dir) / "releases"
for d in releases.iterdir() if releases.exists() else []:
if d.resolve() != keep.resolve() and d.is_dir():
shutil.rmtree(d, ignore_errors=True)
log.info("pruned release %s", d.name[:12])
self.docker.image_prune()
def _compose(self) -> Compose:
return Compose(self.docker, Path(self.cfg.deploy_dir) / "current", self.cfg.compose_project)
# -- misc ----------------------------------------------------------------------------------------------
def _report(
self,
result: str,
sha: str | None,
*,
containers: list[Container] | None = None,
detail: str | None = None,
extra_tail: str = "",
) -> None:
if self.tenancy is None:
return
tail = self.tail.text()
if extra_tail:
tail = tail + "\n--- compose logs ---\n" + REDACTOR.scrub(extra_tail)
try:
self.tenancy.report(
sha=sha,
result=result,
log_tail=tail,
containers=[c.as_dict() for c in containers] if containers else None,
detail=REDACTOR.scrub(detail) if detail else None,
)
self.state.last_report_at = time.time()
self.state.last_result = result
except (TenancyError, TransportError) as exc:
log.warning("report %s not delivered: %s", result, exc)
def _host_facts(self) -> dict:
os_id = "unknown"
try:
kv = dict(line.split("=", 1) for line in Path("/etc/os-release").read_text().splitlines() if "=" in line)
os_id = f"{kv.get('ID', '?').strip(chr(34))}-{kv.get('VERSION_ID', '?').strip(chr(34))}"
except OSError:
pass
facts = {
"hostname": socket.gethostname(),
"os": os_id,
"kernel": platform.release(),
"agent_version": __version__,
"transport": self.cfg.transport,
"site": self.cfg.site,
"laptop_mode": self.cfg.laptop_mode,
}
if self.docker.available():
facts["docker"] = self.docker.version()
facts["compose"] = self.docker.compose_version()
facts["free_bytes"] = self.docker.free_bytes()
return facts
def _scrub(self) -> None:
for k in list(self._values):
self._values[k] = ""
self._values.clear()
for d in self._staging:
shutil.rmtree(d, ignore_errors=True)
self._staging.clear()
def _short(sha: str | None) -> str:
return sha[:12] if sha else "-"
+147
View File
@@ -0,0 +1,147 @@
"""OpenBao over the mesh: jwt-tenancy login, token upkeep, KV-v2 reads of the env's own secrets.
The agent never sees a secret from tenancy: it logs in with the deploy grant
(`POST /v1/auth/<mount>/login {"role": "see-env", "jwt": <grant>}`) and reads
`<kv>/data/<env_id>/see/<name>` itself. The policy `see-env` is templated on the token's
entity alias (`user_claim=env_id`), so a manifest path for another env is refused here
BEFORE it could even be tried (`ManifestPathError`)."""
from __future__ import annotations
import logging
import urllib.parse
from dataclasses import dataclass
from monky_deployd.redact import REDACTOR
from monky_deployd.transport import HttpClient, HttpResponse, TransportError
log = logging.getLogger("monky-deployd.bao")
class BaoError(Exception):
def __init__(self, status: int, errors: list[str] | str, where: str = ""):
errs = errors if isinstance(errors, list) else [str(errors)]
super().__init__(f"{where or 'openbao'}: {status} {'; '.join(errs)[:300]}")
self.status = status
self.errors = errs
class ManifestPathError(Exception):
pass
@dataclass
class BaoToken:
client_token: str
accessor: str | None
ttl_s: int
renewable: bool
meta: dict
policies: list[str]
@property
def grant_jti(self) -> str | None:
return (self.meta or {}).get("grant_jti")
def kv_data_path(kv_mount: str, manifest_path: str, env_id: str) -> str:
"""`monky/<env>/see/<name>` or `monky/data/<env>/see/<name>` -> `/v1/monky/data/<env>/see/<name>`.
The env segment MUST equal this agent's env_id: the agent reads its own subtree and nothing
else (ADR-0028 §4 "what the agent may read")."""
parts = [p for p in manifest_path.strip("/").split("/") if p]
if len(parts) < 3 or parts[0] != kv_mount:
raise ManifestPathError(f"manifest path {manifest_path!r} is not under KV mount {kv_mount!r}")
rest = parts[1:]
if rest[0] == "data":
rest = rest[1:]
if len(rest) < 3 or rest[0] != env_id or rest[1] != "see":
raise ManifestPathError(f"manifest path {manifest_path!r} is outside {kv_mount}/data/{env_id}/see/")
if any(p in (".", "..") or not p for p in rest):
raise ManifestPathError(f"manifest path {manifest_path!r} is malformed")
return "/v1/" + "/".join([kv_mount, "data", *rest])
def _errors(resp: HttpResponse) -> list[str]:
try:
js = resp.json()
if isinstance(js, dict) and isinstance(js.get("errors"), list):
return [str(e) for e in js["errors"]] or [f"http {resp.status}"]
except TransportError:
pass
return [resp.text()[:200] or f"http {resp.status}"]
class BaoClient:
def __init__(self, http: HttpClient, *, auth_mount: str, role: str, kv_mount: str):
self.http = http
self.auth_mount = auth_mount
self.role = role
self.kv_mount = kv_mount
def _call(self, method: str, path: str, token: str | None = None, json_body=None) -> HttpResponse:
headers = {"X-Vault-Request": "true"}
if token:
headers["X-Vault-Token"] = token
resp = self.http.request(method, path, json_body=json_body, headers=headers)
if resp.status >= 400:
raise BaoError(resp.status, _errors(resp), f"{method} {path.split('?')[0]}")
return resp
# -- auth --------------------------------------------------------------------------------------
def login(self, grant_jwt: str, *, mount: str | None = None, role: str | None = None) -> BaoToken:
mount = mount or self.auth_mount
role = role or self.role
REDACTOR.add(grant_jwt)
js = self._call("POST", f"/v1/auth/{mount}/login", json_body={"role": role, "jwt": grant_jwt}).json()
auth = (js or {}).get("auth") or {}
tok = auth.get("client_token")
if not tok:
raise BaoError(200, "login returned no client_token", f"auth/{mount}/login")
REDACTOR.add(tok)
return BaoToken(
client_token=tok,
accessor=auth.get("accessor"),
ttl_s=int(auth.get("lease_duration") or 0),
renewable=bool(auth.get("renewable", False)),
meta=dict(auth.get("metadata") or {}),
policies=list(auth.get("token_policies") or auth.get("policies") or []),
)
def lookup_self(self, token: str) -> dict:
js = self._call("GET", "/v1/auth/token/lookup-self", token).json()
return dict((js or {}).get("data") or {})
def renew_self(self, token: str, increment_s: int | None = None) -> int:
body = {"increment": f"{increment_s}s"} if increment_s else {}
js = self._call("POST", "/v1/auth/token/renew-self", token, json_body=body).json()
return int(((js or {}).get("auth") or {}).get("lease_duration") or 0)
def revoke_self(self, token: str) -> None:
try:
self._call("POST", "/v1/auth/token/revoke-self", token, json_body={})
except BaoError as exc:
log.debug("revoke-self ignored: %s", exc)
# -- KV ----------------------------------------------------------------------------------------
def kv_read(self, token: str, manifest_path: str, env_id: str, version: int | None) -> str:
path = kv_data_path(self.kv_mount, manifest_path, env_id)
if version:
path += "?" + urllib.parse.urlencode({"version": int(version)})
js = self._call("GET", path, token).json() or {}
data = (js.get("data") or {}).get("data")
if not isinstance(data, dict) or not data:
raise BaoError(200, "empty secret", path.split("?")[0])
if "value" in data:
value = data["value"]
elif len(data) == 1:
value = next(iter(data.values()))
else:
raise BaoError(200, "secret has no 'value' key and is ambiguous", path.split("?")[0])
if not isinstance(value, str):
value = str(value)
REDACTOR.add(value)
got = ((js.get("data") or {}).get("metadata") or {}).get("version")
if version and got is not None and int(got) != int(version):
raise BaoError(200, f"version mismatch: wanted {version}, got {got}", path.split("?")[0])
return value
+198
View File
@@ -0,0 +1,198 @@
"""The rendered bundle: parse the tar, verify the sha, and the refusal checks that run
BEFORE any secret is read (so a refused bundle never causes a lease).
Files (monky-deploy `render_files`): `docker-compose.yml`, `.env.template`,
`secrets.manifest.json`, `bundle.json` (+ `.env.example`). The sha is monky-tenancy's
`bundle_sha`: sha256 over sorted (name, "\\0", content, "\\0")."""
from __future__ import annotations
import hashlib
import io
import json
import re
import tarfile
from dataclasses import dataclass, field
COMPOSE_NAMES = ("docker-compose.yml", "docker-compose.yaml", "compose.yaml", "compose.yml")
MANIFEST = "secrets.manifest.json"
ENV_TEMPLATE = ".env.template"
BUNDLE_JSON = "bundle.json"
# ${VAR}, ${VAR:-default}, ${VAR-default}, ${VAR:?err}, ${VAR?err}; `$$` is an escape
_VAR_RE = re.compile(r"(?<!\$)\$\{([A-Za-z_][A-Za-z0-9_]*)(?::?[-?+][^}]*)?\}")
_VAR_DEFAULTED_RE = re.compile(r"(?<!\$)\$\{([A-Za-z_][A-Za-z0-9_]*):?[-+][^}]*\}")
_PRIV_RE = re.compile(r"^\s*privileged\s*:\s*(true|yes|on)\s*$", re.I | re.M)
_HOSTNET_RE = re.compile(r"^\s*network_mode\s*:\s*[\"']?host[\"']?\s*$", re.I | re.M)
_PID_HOST_RE = re.compile(r"^\s*pid\s*:\s*[\"']?host[\"']?\s*$", re.I | re.M)
_CAP_SYSADMIN_RE = re.compile(r"^\s*-\s*[\"']?(ALL|SYS_ADMIN)[\"']?\s*$", re.M)
class BundleError(Exception):
"""A malformed or mismatching bundle (refusal: exit 1, report failed)."""
def __init__(self, code: str, detail: str):
super().__init__(f"{code}: {detail}")
self.code, self.detail = code, detail
@dataclass
class Bundle:
files: dict[str, str]
sha: str
compose_name: str
meta: dict = field(default_factory=dict)
manifest: dict = field(default_factory=dict)
@property
def compose(self) -> str:
return self.files[self.compose_name]
@property
def env_template(self) -> str:
return self.files.get(ENV_TEMPLATE, "")
@property
def env_id(self) -> str | None:
return self.meta.get("env_id")
@property
def tier(self) -> str | None:
return self.meta.get("tier")
@property
def agent_profile(self) -> dict:
prof = self.meta.get("agent") or {}
return prof if isinstance(prof, dict) else {}
def flag(self, name: str) -> bool:
"""`bundle.json.<name>` or `bundle.json.agent.<name>` (either spelling wins)."""
return bool(self.meta.get(name) or self.agent_profile.get(name))
@property
def disk_need_bytes(self) -> int:
for src in (self.agent_profile, self.meta):
for key in ("disk_need_bytes", "need_bytes"):
v = src.get(key)
if isinstance(v, (int, float)) and v > 0:
return int(v)
d = src.get("disk")
if isinstance(d, dict) and isinstance(d.get("need_bytes"), (int, float)):
return int(d["need_bytes"])
return 0
def bundle_sha(files: dict[str, str]) -> str:
h = hashlib.sha256()
for name in sorted(files):
h.update(name.encode())
h.update(b"\0")
h.update(files[name].encode())
h.update(b"\0")
return h.hexdigest()
def parse(data: bytes, *, max_bytes: int = 4 * 1024 * 1024) -> Bundle:
if len(data) > max_bytes:
raise BundleError("BUNDLE_TOO_LARGE", f"{len(data)} bytes")
files: dict[str, str] = {}
try:
with tarfile.open(fileobj=io.BytesIO(data), mode="r:*") as tar:
for m in tar.getmembers():
if not m.isfile():
if m.isdir():
continue
raise BundleError("BUNDLE_MEMBER", f"{m.name!r} is not a regular file")
name = m.name
while name.startswith("./"):
name = name[2:]
if not name or name.startswith("/") or ".." in name.split("/"):
raise BundleError("BUNDLE_MEMBER", f"unsafe member name {m.name!r}")
fh = tar.extractfile(m)
raw = fh.read() if fh else b""
try:
files[name] = raw.decode("utf-8")
except UnicodeDecodeError as exc:
raise BundleError("BUNDLE_MEMBER", f"{name} is not UTF-8 text") from exc
except tarfile.TarError as exc:
raise BundleError("BUNDLE_TAR", str(exc)) from exc
compose_name = next((n for n in COMPOSE_NAMES if n in files), None)
if compose_name is None:
raise BundleError("BUNDLE_INCOMPLETE", f"no compose file among {list(files)}")
meta: dict = {}
if BUNDLE_JSON in files:
try:
meta = json.loads(files[BUNDLE_JSON])
except json.JSONDecodeError as exc:
raise BundleError("BUNDLE_JSON", str(exc)) from exc
manifest: dict = {"vault_mode": "openbao", "entries": []}
if MANIFEST in files:
try:
manifest = json.loads(files[MANIFEST])
except json.JSONDecodeError as exc:
raise BundleError("BUNDLE_MANIFEST", str(exc)) from exc
if not isinstance(manifest.get("entries"), list):
raise BundleError("BUNDLE_MANIFEST", "entries must be a list")
for e in manifest["entries"]:
if not isinstance(e, dict) or not e.get("var") or not e.get("path"):
raise BundleError("BUNDLE_MANIFEST", f"bad entry {e!r}")
if "value" in e:
raise BundleError("BUNDLE_MANIFEST", f"entry {e['var']} carries a value (refused)")
return Bundle(files=files, sha=bundle_sha(files), compose_name=compose_name, meta=meta, manifest=manifest)
# --- refusal checks (pure; names only, never values) -----------------------------------------
def referenced_vars(text: str) -> set[str]:
return {m.group(1) for m in _VAR_RE.finditer(text)}
def defaulted_vars(text: str) -> set[str]:
return {m.group(1) for m in _VAR_DEFAULTED_RE.finditer(text)}
def unresolved_vars(bundle: Bundle, provided: set[str]) -> list[str]:
"""Variables the compose file / .env.template need that neither the manifest nor a compose
default supplies. NAMES only."""
need = referenced_vars(bundle.compose) | referenced_vars(bundle.env_template)
have = provided | defaulted_vars(bundle.compose)
return sorted(need - have)
def privileged_findings(bundle: Bundle) -> list[str]:
text = bundle.compose
out = []
if _PRIV_RE.search(text):
out.append("privileged: true")
if _HOSTNET_RE.search(text):
out.append("network_mode: host")
if _PID_HOST_RE.search(text):
out.append("pid: host")
if re.search(r"^\s*cap_add\s*:", text, re.M) and _CAP_SYSADMIN_RE.search(text):
out.append("cap_add SYS_ADMIN/ALL")
return out
def manifest_vars(bundle: Bundle) -> set[str]:
return {e["var"] for e in bundle.manifest.get("entries", [])}
def render_env(template: str, values: dict[str, str]) -> str:
"""Fill `VAR=${VAR}` lines; every other line is copied. Values are quoted the compose way
(double quotes, `\\`/`"` escaped, `$` doubled) unless plain."""
out = []
for line in template.splitlines():
m = re.match(r"^([A-Za-z_][A-Za-z0-9_]*)=\$\{\1\}\s*$", line)
if m and m.group(1) in values:
out.append(f"{m.group(1)}={_quote(values[m.group(1)])}")
else:
out.append(line)
return "\n".join(out) + "\n"
def _quote(v: str) -> str:
if re.fullmatch(r"[A-Za-z0-9_./:@+=,%~-]*", v):
return v
esc = v.replace("\\", "\\\\").replace('"', '\\"').replace("$", "$$").replace("\n", "\\n")
return f'"{esc}"'
+217
View File
@@ -0,0 +1,217 @@
"""`monky-deployd run [--once] [--prune]` · `status [--json]` · `bootstrap` · `version`."""
from __future__ import annotations
import argparse
import json
import logging
import os
import random
import signal
import sys
import time
from pathlib import Path
from monky_deployd import __version__
from monky_deployd import state as statemod
from monky_deployd.config import DEFAULT_CONFIG_PATH, Config, ConfigError, load
from monky_deployd.redact import install as install_redactor
EX_OK, EX_FAIL, EX_TEMPFAIL, EX_ENV_MISMATCH = 0, 1, 75, 78
_PRIO = {"DEBUG": 7, "INFO": 6, "WARNING": 4, "ERROR": 3, "CRITICAL": 2}
class _JournalFormatter(logging.Formatter):
"""sd-daemon priority prefix; journald strips it and files the level (no timestamp: journald has one)."""
def format(self, record: logging.LogRecord) -> str:
return f"<{_PRIO.get(record.levelname, 6)}>{record.name}: {record.getMessage()}"
def setup_logging(level: str = "INFO") -> None:
root = logging.getLogger()
root.handlers.clear()
h = logging.StreamHandler(sys.stderr)
if os.environ.get("JOURNAL_STREAM"):
h.setFormatter(_JournalFormatter())
else:
h.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(name)s: %(message)s"))
root.addHandler(h)
root.setLevel(getattr(logging, level.upper(), logging.INFO))
install_redactor(root)
def _load(args) -> Config:
cfg = load(args.config)
if args.verbose:
cfg.log_level = "DEBUG"
return cfg
def cmd_run(args) -> int:
cfg = _load(args)
setup_logging(cfg.log_level)
from monky_deployd.agent import Agent
if args.once:
return Agent(cfg, prune=args.prune).run_once()
stop = {"now": False}
def _sig(*_):
stop["now"] = True
signal.signal(signal.SIGTERM, _sig)
signal.signal(signal.SIGINT, _sig)
log = logging.getLogger("monky-deployd")
log.info("loop mode (interval %ss%s)", cfg.interval_s, ", laptop" if cfg.laptop_mode else "")
rc = EX_OK
while not stop["now"]:
rc = Agent(cfg, prune=args.prune).run_once()
if rc == EX_ENV_MISMATCH:
return rc # no retry storm
delay = cfg.interval_s + random.uniform(0, min(10, cfg.interval_s / 6))
for _ in range(int(delay)):
if stop["now"]:
break
time.sleep(1)
return rc
def cmd_bootstrap(args) -> int:
cfg = _load(args)
setup_logging(cfg.log_level)
from monky_deployd.agent import Agent, NoCredentials
from monky_deployd.bao import BaoError
from monky_deployd.tenancy import EnvMismatch
from monky_deployd.transport import TransportError
agent = Agent(cfg)
if cfg.token_path.exists() and not args.force:
print(f"already bootstrapped ({cfg.token_path} exists); use --force to log in again")
return EX_OK
if args.force:
statemod.delete(cfg.token_path)
try:
agent.state = statemod.load(cfg.state_path, cfg.env_id)
agent._ensure_token()
statemod.save(cfg.state_path, agent.state)
except TransportError as exc:
print(f"network: {exc}", file=sys.stderr)
return EX_TEMPFAIL
except EnvMismatch as exc:
print(f"AGENT_ENV_MISMATCH: {exc.detail}", file=sys.stderr)
return EX_ENV_MISMATCH
except (NoCredentials, BaoError) as exc:
print(f"bootstrap failed: {exc}", file=sys.stderr)
return EX_FAIL
print(f"bootstrapped: token stored at {cfg.token_path} (accessor {agent.state.token.accessor})")
return EX_OK
def _fmt_ts(ts: float | None) -> str:
if not ts:
return "-"
return time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(ts)) + f" ({int(time.time() - ts)}s ago)"
def cmd_status(args) -> int:
cfg = _load(args)
st = statemod.load(cfg.state_path, cfg.env_id)
token_present = cfg.token_path.exists()
bootstrap_present = Path(cfg.bootstrap_path).exists()
current = Path(cfg.deploy_dir) / "current"
containers: list[dict] = []
try:
from monky_deployd.compose import Compose, Docker
d = Docker(cfg.docker_bin)
if d.available() and current.exists():
containers = [c.as_dict() for c in Compose(d, current, cfg.compose_project).ps()]
except Exception: # status must never crash on docker trouble
containers = []
healthy = bool(containers) and all(
c["state"] == "running" and c["health"] in ("", "healthy", "none") for c in containers
)
out = {
"env_id": cfg.env_id,
"site": cfg.site,
"transport": cfg.transport,
"agent_version": __version__,
"laptop_mode": cfg.laptop_mode,
"token_present": token_present,
"bootstrap_grant_present": bootstrap_present,
"token": {"accessor": st.token.accessor, "issued_at": st.token.issued_at, "source": st.token.source},
"applied_sha": st.applied_sha,
"desired_sha": st.desired_sha,
"in_sync": bool(st.applied_sha) and st.applied_sha == st.desired_sha,
"last_checkin_at": st.last_checkin_at,
"last_report_at": st.last_report_at,
"last_action": st.last_action,
"last_result": st.last_result,
"last_error": st.last_error,
"current_release": os.path.realpath(current) if current.exists() else None,
"containers": containers,
"healthy": healthy,
}
if args.json:
print(json.dumps(out, indent=2, sort_keys=True))
return EX_OK
print(
f"monky-deployd {__version__}{cfg.env_id} ({cfg.site}, transport {cfg.transport}{', laptop' if cfg.laptop_mode else ''})"
)
print(
f" credentials : token {'present' if token_present else 'ABSENT'}"
f"{' (accessor ' + st.token.accessor + ')' if st.token.accessor else ''}"
f"{'; bootstrap grant waiting' if bootstrap_present else ''}"
)
print(f" applied sha : {st.applied_sha or '-'}")
print(f" desired sha : {st.desired_sha or '-'} {'in sync' if out['in_sync'] else 'NOT in sync'}")
print(f" last checkin: {_fmt_ts(st.last_checkin_at)} action={st.last_action or '-'}")
print(f" last report : {_fmt_ts(st.last_report_at)} result={st.last_result or '-'}")
if st.last_error:
print(f" last error : {st.last_error}")
print(f" release : {out['current_release'] or '-'}")
if containers:
print(f" containers : {'healthy' if healthy else 'UNHEALTHY'}")
for c in containers:
print(f" - {c['name']}: {c['state']} {c['health'] or ''}".rstrip())
else:
print(" containers : none (docker unavailable or nothing deployed)")
return EX_OK if (token_present and (out["in_sync"] or not st.desired_sha)) else EX_FAIL
def cmd_version(_args) -> int:
print(__version__)
return EX_OK
def build_parser() -> argparse.ArgumentParser:
p = argparse.ArgumentParser(prog="monky-deployd", description="Monky backend pull agent (ADR-0028)")
p.add_argument("-c", "--config", default=os.environ.get("MONKY_DEPLOYD_CONFIG", DEFAULT_CONFIG_PATH))
p.add_argument("-v", "--verbose", action="store_true")
sub = p.add_subparsers(dest="cmd", required=True)
r = sub.add_parser("run", help="reconcile: --once for the timer/oneshot, otherwise loop")
r.add_argument("--once", action="store_true")
r.add_argument(
"--prune", action="store_true", help="after a successful apply, drop old releases + docker image prune"
)
r.set_defaults(fn=cmd_run)
s = sub.add_parser("status", help="what this box has applied vs what tenancy wants")
s.add_argument("--json", action="store_true")
s.set_defaults(fn=cmd_status)
b = sub.add_parser("bootstrap", help="log in to OpenBao with the install kit's grant; store the token")
b.add_argument("--force", action="store_true")
b.set_defaults(fn=cmd_bootstrap)
sub.add_parser("version").set_defaults(fn=cmd_version)
return p
def main(argv: list[str] | None = None) -> int:
args = build_parser().parse_args(argv)
try:
return int(args.fn(args))
except ConfigError as exc:
print(f"config error: {exc}", file=sys.stderr)
return EX_ENV_MISMATCH if "env_id" in str(exc) else EX_FAIL
except KeyboardInterrupt:
return 130
+224
View File
@@ -0,0 +1,224 @@
"""`docker compose` + a few `docker` facts. Everything shells out with a timeout and captured
output; nothing here ever sees a secret except the `.env` file compose reads by itself."""
from __future__ import annotations
import json
import logging
import os
import shutil
import subprocess
import time
from dataclasses import dataclass
from pathlib import Path
log = logging.getLogger("monky-deployd.compose")
class ComposeError(Exception):
def __init__(self, what: str, rc: int, output: str):
super().__init__(f"{what} failed (rc={rc})")
self.what, self.rc, self.output = what, rc, output
@dataclass
class Container:
name: str
state: str
health: str
exit_code: int | None = None
def as_dict(self) -> dict:
return {"name": self.name, "state": self.state, "health": self.health}
@property
def ok(self) -> bool:
if self.state == "running":
return self.health in ("", "healthy", "none")
if self.state == "exited":
return (self.exit_code or 0) == 0
return False
class Docker:
def __init__(self, docker_bin: str = "docker", timeout_s: int = 600):
self.bin = docker_bin
self.timeout_s = timeout_s
def available(self) -> bool:
return shutil.which(self.bin) is not None
def run(self, args: list[str], *, cwd: str | None = None, timeout: int | None = None, check: bool = True) -> str:
cmd = [self.bin, *args]
what = _what(cmd)
log.debug("exec: %s", " ".join(cmd))
try:
p = subprocess.run(
cmd,
cwd=cwd,
capture_output=True,
text=True,
timeout=timeout or self.timeout_s,
env={**os.environ, "COMPOSE_INTERACTIVE_NO_CLI": "1"},
)
except FileNotFoundError as exc:
raise ComposeError(what, 127, f"{self.bin} not found") from exc
except subprocess.TimeoutExpired as exc:
raise ComposeError(what, 124, f"timeout after {exc.timeout}s") from exc
out = (p.stdout or "") + (("\n" + p.stderr) if p.stderr else "")
if check and p.returncode != 0:
raise ComposeError(what, p.returncode, out.strip())
return out
# -- facts ------------------------------------------------------------------------------------
def version(self) -> str | None:
try:
return self.run(["version", "--format", "{{.Server.Version}}"], timeout=20).strip() or None
except ComposeError:
return None
def compose_version(self) -> str | None:
try:
return self.run(["compose", "version", "--short"], timeout=20).strip() or None
except ComposeError:
return None
def data_root(self) -> str:
try:
root = self.run(["info", "--format", "{{.DockerRootDir}}"], timeout=20).strip()
except ComposeError:
root = ""
return root or "/var/lib/docker"
def free_bytes(self, path: str | None = None) -> int | None:
p = path or self.data_root()
while p and not os.path.exists(p):
p = os.path.dirname(p)
try:
st = os.statvfs(p or "/")
except OSError:
return None
return st.f_bavail * st.f_frsize
def image_prune(self) -> None:
try:
self.run(["image", "prune", "-f"], timeout=300)
except ComposeError as exc:
log.warning("image prune failed: %s", exc)
class Compose:
def __init__(self, docker: Docker, project_dir: Path, project_name: str):
self.docker = docker
self.project_dir = Path(project_dir)
self.project_name = project_name
def _base(self) -> list[str]:
return [
"compose",
"--project-name",
self.project_name,
"--project-directory",
str(self.project_dir),
]
def _with_files(self) -> list[str]:
args = self._base()
env_file = self.project_dir / ".env"
if env_file.exists():
args += ["--env-file", str(env_file)]
return args
def config_check(self) -> str:
return self.docker.run([*self._with_files(), "config", "--quiet"], cwd=str(self.project_dir), timeout=120)
def pull(self) -> str:
return self.docker.run([*self._with_files(), "pull", "--quiet"], cwd=str(self.project_dir))
def up(self) -> str:
return self.docker.run(
[*self._with_files(), "up", "-d", "--remove-orphans", "--quiet-pull"], cwd=str(self.project_dir)
)
def down(self, *, purge_volumes: bool) -> str:
args = [*self._base(), "down", "--remove-orphans"]
if purge_volumes:
args.append("-v")
return self.docker.run(args, cwd=str(self.project_dir) if self.project_dir.exists() else None, timeout=600)
def ps(self) -> list[Container]:
if not self.project_dir.exists():
return []
try:
out = self.docker.run(
[*self._base(), "ps", "-a", "--format", "json"], cwd=str(self.project_dir), timeout=60
)
except ComposeError as exc:
log.warning("compose ps failed: %s", exc)
return []
return parse_ps(out)
def logs_tail(self, lines: int = 80) -> str:
try:
return self.docker.run(
[*self._base(), "logs", "--no-color", "--tail", str(lines)], cwd=str(self.project_dir), timeout=60
)
except ComposeError as exc:
return exc.output
def wait_healthy(self, timeout_s: int, poll_s: float = 5.0) -> tuple[bool, list[Container]]:
deadline = time.monotonic() + timeout_s
containers: list[Container] = []
while True:
containers = self.ps()
if containers and all(c.ok for c in containers) and not any(c.health == "starting" for c in containers):
return True, containers
if any(c.state == "exited" and (c.exit_code or 0) != 0 for c in containers):
return False, containers
if time.monotonic() >= deadline:
return False, containers
time.sleep(poll_s)
def _what(cmd: list[str]) -> str:
"""`docker compose <sub>` / `docker <sub>` for error messages (compose global flags skipped)."""
if len(cmd) > 1 and cmd[1] == "compose":
rest = cmd[2:]
while rest and rest[0].startswith("--"):
rest = rest[2:]
return "docker compose " + (rest[0] if rest else "")
return " ".join(cmd[:2])
def parse_ps(out: str) -> list[Container]:
"""`compose ps --format json` is NDJSON on compose >= 2.21 and a JSON array before."""
text = out.strip()
if not text:
return []
rows: list[dict] = []
if text.startswith("["):
try:
rows = json.loads(text)
except json.JSONDecodeError:
rows = []
else:
for line in text.splitlines():
line = line.strip()
if line.startswith("{"):
try:
rows.append(json.loads(line))
except json.JSONDecodeError:
continue
out_list = []
for r in rows:
state = str(r.get("State") or r.get("state") or "").lower()
health = str(r.get("Health") or r.get("health") or "").lower()
code = r.get("ExitCode", r.get("exit_code"))
try:
code = int(code) if code is not None else None
except (TypeError, ValueError):
code = None
out_list.append(
Container(name=str(r.get("Name") or r.get("name") or "?"), state=state, health=health, exit_code=code)
)
return out_list
+294
View File
@@ -0,0 +1,294 @@
"""`/etc/monky-deployd/config.yaml` — parsed with a small YAML *subset* reader (stdlib only).
Supported: nested block maps by indentation, `key: value` scalars (str / int / float / bool /
null, quoted strings), `- item` lists of scalars, `#` comments. That is exactly what the
install kit and the ansible role write. Anything fancier (anchors, flow style, multi-line
scalars) is a config error, not a silent misread."""
from __future__ import annotations
import os
import re
from dataclasses import dataclass, field
from pathlib import Path
DEFAULT_CONFIG_PATH = "/etc/monky-deployd/config.yaml"
TRANSPORTS = ("sdk", "proxy", "system")
ENV_ID_RE = re.compile(r"^env-(dev|qa|stage|prod)-[0-9]{2,3}$|^(dev-env-2|prod-cedar)$")
SITES = ("cbs", "pdx")
class ConfigError(Exception):
pass
# --- YAML subset -----------------------------------------------------------------------------
def _scalar(raw: str):
s = raw.strip()
if s == "" or s in ("~", "null", "Null", "NULL"):
return None
if len(s) >= 2 and s[0] == s[-1] and s[0] in "\"'":
inner = s[1:-1]
if s[0] == '"':
inner = inner.encode().decode("unicode_escape")
return inner
low = s.lower()
if low in ("true", "yes", "on"):
return True
if low in ("false", "no", "off"):
return False
if re.fullmatch(r"[+-]?[0-9]+", s):
return int(s)
if re.fullmatch(r"[+-]?[0-9]*\.[0-9]+", s):
return float(s)
return s
def _strip_comment(line: str) -> str:
out, quote = [], None
for i, ch in enumerate(line):
if quote:
if ch == quote:
quote = None
elif ch in "\"'":
quote = ch
elif ch == "#" and (i == 0 or line[i - 1] in " \t"):
break
out.append(ch)
return "".join(out).rstrip()
def parse_yaml_subset(text: str) -> dict:
lines: list[tuple[int, str]] = []
for n, raw in enumerate(text.splitlines(), 1):
if raw.strip().startswith("#") or not raw.strip():
continue
if raw.strip() in ("---", "..."):
continue
if "\t" in raw[: len(raw) - len(raw.lstrip())]:
raise ConfigError(f"line {n}: tabs are not allowed for indentation")
body = _strip_comment(raw)
if not body.strip():
continue
lines.append((len(body) - len(body.lstrip(" ")), body.strip()))
pos = 0
def block(indent: int):
nonlocal pos
if pos < len(lines) and lines[pos][1].startswith("- "):
return seq(indent)
return mapping(indent)
def seq(indent: int) -> list:
nonlocal pos
items = []
while pos < len(lines) and lines[pos][0] == indent and lines[pos][1].startswith("- "):
items.append(_scalar(lines[pos][1][2:]))
pos += 1
return items
def mapping(indent: int) -> dict:
nonlocal pos
out: dict = {}
while pos < len(lines) and lines[pos][0] == indent:
ind, body = lines[pos]
m = re.match(r"^([A-Za-z0-9_.\-]+)\s*:(?:\s+(.*)|$)", body)
if not m:
raise ConfigError(f"cannot parse: {body!r}")
key, val = m.group(1), m.group(2)
pos += 1
if val is None or val == "":
if pos < len(lines) and lines[pos][0] > ind:
out[key] = block(lines[pos][0])
else:
out[key] = None
else:
if val.startswith("[") or val.startswith("{"):
raise ConfigError(f"flow style is not supported ({key})")
out[key] = _scalar(val)
if pos < len(lines) and lines[pos][0] > indent:
raise ConfigError(f"unexpected indentation near {lines[pos][1]!r}")
return out
if not lines:
return {}
result = block(lines[0][0])
if pos != len(lines):
raise ConfigError(f"unexpected content near {lines[pos][1]!r}")
if not isinstance(result, dict):
raise ConfigError("top level must be a map")
return result
# --- the config -------------------------------------------------------------------------------
@dataclass
class TenancyCfg:
service: str = "monky.tenancy.deploy"
host: str = "" # intercept host; defaults to `service`
port: int = 8081
scheme: str = "http"
proxy_addr: str = "127.0.0.1:18443"
timeout_s: int = 30
@dataclass
class BaoCfg:
service: str = "openbao"
addr: str = "https://bao.cbs.tikali.net:8200"
proxy_addr: str = "127.0.0.1:18200"
ca_bundle: str | None = "/etc/monky-deployd/openbao-ca.pem"
mount: str = "jwt-tenancy" # the AUTH mount (Gate 1 v2, ADR-0028 amendment)
role: str = "see-env"
kv_mount: str = "monky"
token_max_ttl_s: int = 30 * 86400
renew_below_s: int = 12 * 3600
release_before_s: int = 2 * 86400
timeout_s: int = 30
@dataclass
class DiskCfg:
factor: float = 1.5
headroom_bytes: int = 2 * 1024**3
@dataclass
class Config:
env_id: str
site: str
transport: str = "sdk"
identity: str = ""
tenancy: TenancyCfg = field(default_factory=TenancyCfg)
bao: BaoCfg = field(default_factory=BaoCfg)
disk: DiskCfg = field(default_factory=DiskCfg)
state_dir: str = "/var/lib/monky-deployd"
deploy_dir: str = ""
bootstrap_path: str = "/etc/monky-deployd/bootstrap.jwt"
interval_s: int = 60
volumes_on_absent: str = "keep"
laptop_mode: bool = False
prod: bool | None = None
healthy_timeout_s: int = 300
compose_project: str = ""
docker_bin: str = "docker"
log_level: str = "INFO"
path: str = DEFAULT_CONFIG_PATH
# derived
@property
def token_path(self) -> Path:
return Path(self.state_dir) / "bao.token"
@property
def state_path(self) -> Path:
return Path(self.state_dir) / "state.json"
@property
def lock_path(self) -> Path:
return Path(self.state_dir) / "lock"
@property
def is_prod(self) -> bool:
if self.prod is not None:
return self.prod
return self.env_id.startswith("env-prod-") or self.env_id == "prod-cedar"
@property
def bao_url(self) -> tuple[str, str, int]:
scheme, rest = self.bao.addr.split("://", 1)
hostport = rest.split("/", 1)[0]
if hostport.startswith("["):
host, _, port = hostport[1:].partition("]")
port = port.lstrip(":")
else:
host, _, port = hostport.partition(":")
return scheme, host, int(port or (443 if scheme == "https" else 80))
def _apply(obj, data: dict, section: str) -> None:
for k, v in (data or {}).items():
key = k.replace("-", "_")
if not hasattr(obj, key):
raise ConfigError(f"unknown key {section}.{k}")
setattr(obj, key, v)
def from_dict(data: dict, path: str = DEFAULT_CONFIG_PATH) -> Config:
data = dict(data or {})
if not data.get("env_id"):
raise ConfigError("env_id is required")
if not data.get("site"):
raise ConfigError("site is required")
cfg = Config(env_id=str(data.pop("env_id")), site=str(data.pop("site")), path=path)
for section, obj in (("tenancy", cfg.tenancy), ("bao", cfg.bao), ("disk", cfg.disk)):
sub = data.pop(section, None)
if sub is None:
continue
if not isinstance(sub, dict):
raise ConfigError(f"{section} must be a map")
# tolerated aliases
if section == "bao":
sub = dict(sub)
if "auth_mount" in sub:
sub["mount"] = sub.pop("auth_mount")
if "approle" in sub:
raise ConfigError(
"bao.approle is not supported: monky-deployd logs in to the jwt-tenancy "
"mount with a deploy grant (ADR-0028 amendment 2026-09-05)"
)
if "base_url" in sub:
sub.pop("base_url")
if section == "tenancy":
sub = dict(sub)
base = sub.pop("base_url", None)
if base:
scheme, rest = base.split("://", 1)
host, _, port = rest.rstrip("/").partition(":")
sub.setdefault("scheme", scheme)
sub.setdefault("host", host)
if port:
sub.setdefault("port", int(port))
_apply(obj, sub, section)
_apply(cfg, data, "")
return validate(cfg)
def validate(cfg: Config) -> Config:
if not ENV_ID_RE.match(cfg.env_id):
raise ConfigError(f"env_id {cfg.env_id!r} is not env-<tier>-<nn> (or a grandfathered id)")
cfg.site = cfg.site.lower()
if cfg.site not in SITES:
raise ConfigError(f"site must be one of {SITES}")
if cfg.transport not in TRANSPORTS:
raise ConfigError(f"transport must be one of {TRANSPORTS}")
if cfg.transport == "sdk" and not cfg.identity:
cfg.identity = f"/opt/openziti/etc/identities/monky-host.{cfg.env_id}.json"
if cfg.volumes_on_absent not in ("keep", "purge"):
raise ConfigError("volumes_on_absent must be keep|purge")
if not cfg.tenancy.host:
cfg.tenancy.host = cfg.tenancy.service
if not cfg.deploy_dir:
cfg.deploy_dir = os.path.join(cfg.state_dir, cfg.env_id)
if not cfg.compose_project:
cfg.compose_project = f"monky-{cfg.env_id}"
if cfg.interval_s < 10:
raise ConfigError("interval_s must be >= 10")
if cfg.disk.factor < 1.0:
raise ConfigError("disk.factor must be >= 1.0")
if cfg.bao.ca_bundle in ("", "none"):
cfg.bao.ca_bundle = None
cfg.log_level = str(cfg.log_level).upper()
return cfg
def load(path: str | os.PathLike = DEFAULT_CONFIG_PATH) -> Config:
p = Path(path)
try:
text = p.read_text()
except FileNotFoundError as exc:
raise ConfigError(f"config {p} not found") from exc
return from_dict(parse_yaml_subset(text), str(p))
+77
View File
@@ -0,0 +1,77 @@
"""Log redaction: token shapes + every secret value the agent has seen.
The agent's journal must never carry a bearer, a deploy grant or a `.env` value. `Redactor`
is a `logging.Filter` installed on every handler; `add()` registers a value the moment it is
read from OpenBao (or minted by OpenBao), and `scrub()` is also applied to the `log_tail`
sent to tenancy. Secret NAMES are fine; values never."""
from __future__ import annotations
import logging
import re
import threading
# Token / credential shapes redacted even before a value is known to us.
_SHAPES = [
re.compile(r"\bhv[sbr]\.[A-Za-z0-9_\-]{20,}"), # OpenBao/Vault service, batch, recovery
re.compile(r"\bs\.[A-Za-z0-9]{24,}\b"), # legacy vault token
re.compile(r"\beyJ[A-Za-z0-9_\-]{8,}\.[A-Za-z0-9_\-]{8,}\.[A-Za-z0-9_\-]{8,}"), # JWT
re.compile(r"(?i)(authorization:\s*bearer\s+)\S+"),
re.compile(r"(?i)(x-vault-token:\s*)\S+"),
]
MASK = "[REDACTED]"
class Redactor(logging.Filter):
def __init__(self) -> None:
super().__init__("monky-deployd.redactor")
self._values: set[str] = set()
self._lock = threading.Lock()
def add(self, value: str | None) -> None:
"""Register a secret value (min 4 chars — shorter ones would shred the log)."""
if value and len(value) >= 4:
with self._lock:
self._values.add(value)
def forget_all(self) -> None:
with self._lock:
self._values.clear()
def scrub(self, text: str) -> str:
if not text:
return text
for pat in _SHAPES:
if pat.groups:
text = pat.sub(lambda m: m.group(1) + MASK, text)
else:
text = pat.sub(MASK, text)
with self._lock:
values = sorted(self._values, key=len, reverse=True)
for v in values:
if v in text:
text = text.replace(v, MASK)
return text
# logging.Filter
def filter(self, record: logging.LogRecord) -> bool:
try:
msg = record.getMessage()
except Exception: # pragma: no cover - a broken format string is not our problem
msg = str(record.msg)
record.msg = self.scrub(msg)
record.args = ()
return True
REDACTOR = Redactor()
def install(logger: logging.Logger | None = None) -> Redactor:
"""Attach the singleton to every handler of `logger` (root by default)."""
logger = logger or logging.getLogger()
for h in logger.handlers:
if REDACTOR not in h.filters:
h.addFilter(REDACTOR)
return REDACTOR
+155
View File
@@ -0,0 +1,155 @@
"""On-disk state under /var/lib/monky-deployd (0700): state.json, bao.token, the flock.
Everything is written atomically (tmp + fsync + rename) with mode 0600; the token file is the
ONLY credential the agent keeps, and it is the agent's bearer to tenancy (ADR-0028 §C.C)."""
from __future__ import annotations
import fcntl
import json
import os
import time
from dataclasses import asdict, dataclass, field
from pathlib import Path
HISTORY_MAX = 20
@dataclass
class TokenMeta:
accessor: str | None = None
issued_at: float | None = None
ttl_s: int | None = None
renewable: bool | None = None
grant_jti: str | None = None
source: str | None = None # bootstrap | lease
@dataclass
class State:
env_id: str
applied_sha: str | None = None
applied_at: float | None = None
last_checkin_at: float | None = None
last_report_at: float | None = None
last_result: str | None = None
last_error: str | None = None
last_action: str | None = None
desired_sha: str | None = None
history: list[str] = field(default_factory=list)
token: TokenMeta = field(default_factory=TokenMeta)
agent_version: str | None = None
updated_at: float | None = None
consecutive_failures: int = 0
def remember_applied(self, sha: str) -> None:
self.applied_sha = sha
self.applied_at = time.time()
if sha in self.history:
self.history.remove(sha)
self.history.append(sha)
del self.history[:-HISTORY_MAX]
def is_rollback(self, sha: str) -> bool:
"""`sha` was applied before AND something newer has been applied since."""
if sha not in self.history:
return False
return self.history.index(sha) < len(self.history) - 1
def write_private(path: Path, data: bytes) -> None:
path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
tmp = path.with_name(f".{path.name}.{os.getpid()}.tmp")
fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
try:
with os.fdopen(fd, "wb") as fh:
fh.write(data)
fh.flush()
os.fsync(fh.fileno())
os.chmod(tmp, 0o600)
os.replace(tmp, path)
except BaseException:
try:
os.unlink(tmp)
except FileNotFoundError:
pass
raise
def load(path: Path, env_id: str) -> State:
try:
raw = json.loads(path.read_text())
except FileNotFoundError:
return State(env_id=env_id)
except (OSError, json.JSONDecodeError):
return State(env_id=env_id, last_error="state.json unreadable; reset")
tok = raw.pop("token", None) or {}
known = {k for k in State.__dataclass_fields__}
st = State(**{k: v for k, v in raw.items() if k in known and k != "token"})
st.token = TokenMeta(**{k: v for k, v in tok.items() if k in TokenMeta.__dataclass_fields__})
if st.env_id != env_id:
# state belongs to another env: never mix histories
return State(env_id=env_id, last_error=f"state.json was for {st.env_id}; reset")
return st
def save(path: Path, st: State) -> None:
st.updated_at = time.time()
write_private(path, (json.dumps(asdict(st), indent=2, sort_keys=True) + "\n").encode())
def read_token(path: Path) -> str | None:
try:
tok = path.read_text().strip()
except FileNotFoundError:
return None
return tok or None
def write_token(path: Path, token: str) -> None:
write_private(path, token.encode())
def delete(path: Path) -> bool:
"""Remove a credential file. If the directory is not ours to write (/etc/monky-deployd is
root-owned), truncating the file to zero bytes is just as final: an empty grant is no grant."""
try:
path.unlink()
return True
except FileNotFoundError:
return False
except PermissionError:
try:
with open(path, "wb"):
pass
return True
except OSError:
return False
class Lock:
"""Non-blocking flock; a second concurrent tick exits quietly (the timer will fire again)."""
def __init__(self, path: Path):
self.path = path
self._fd: int | None = None
def acquire(self) -> bool:
self.path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
self._fd = os.open(self.path, os.O_RDWR | os.O_CREAT, 0o600)
try:
fcntl.flock(self._fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
os.close(self._fd)
self._fd = None
return False
os.write(self._fd, str(os.getpid()).encode())
return True
def release(self) -> None:
if self._fd is not None:
try:
fcntl.flock(self._fd, fcntl.LOCK_UN)
finally:
os.close(self._fd)
self._fd = None
+189
View File
@@ -0,0 +1,189 @@
"""The four agent calls on monky-tenancy's agent entrypoint (`/v1/agent/*`, MONKY-ADR-0028 §C.C).
Bearer = the agent's OpenBao token (from the `jwt-tenancy` login). Tenancy pins every call to
the token's `meta.env_id` (403 AGENT_ENV_MISMATCH -> exit 78, never retried) and refuses a token
whose deploy grant was superseded (401 AGENT_UNAUTHENTICATED -> re-bootstrap or re-run the kit).
Lease shape of record (Gate 1 v2, 2026-09-05): `{login_jwt, ttl_s, mount, role, vault}`. An
AppRole-era body (`wrapping_token`, `role_id`) is refused loudly — there is nothing to unwrap."""
from __future__ import annotations
import logging
from dataclasses import dataclass, field
from monky_deployd.transport import HttpClient, HttpResponse, TransportError
log = logging.getLogger("monky-deployd.tenancy")
LOG_TAIL_MAX = 16 * 1024
class TenancyError(Exception):
def __init__(self, status: int, code: str, detail: str = ""):
super().__init__(f"{status} {code}: {detail}".rstrip(": "))
self.status, self.code, self.detail = status, code, detail
class Unauthenticated(TenancyError):
"""401 — the bearer is unknown, expired, revoked or its grant was superseded."""
class EnvMismatch(TenancyError):
"""403 AGENT_ENV_MISMATCH — this box's token is pinned to another env. Exit 78."""
class RateLimited(TenancyError):
def __init__(self, status: int, code: str, detail: str, retry_after: int):
super().__init__(status, code, detail)
self.retry_after = retry_after
class LeaseShapeUnsupported(TenancyError):
"""Tenancy answered with an AppRole lease; this agent only speaks the jwt-tenancy grant."""
@dataclass
class Vault:
addr: str | None = None
mount: str | None = None # the KV mount ("monky")
prefix: str | None = None # "<env>/see"
@dataclass
class Checkin:
env_id: str
action: str
desired_sha: str | None
purge_volumes: bool
bundle_url: str | None
checkin_interval_s: int
vault: Vault = field(default_factory=Vault)
@dataclass
class Lease:
login_jwt: str
ttl_s: int
mount: str
role: str
vault: Vault = field(default_factory=Vault)
def _error(resp: HttpResponse) -> TenancyError:
code, detail = "HTTP_ERROR", resp.text()[:200]
try:
js = resp.json()
if isinstance(js, dict):
code = str(js.get("code") or code)
detail = str(js.get("detail") or detail)
except TransportError:
pass
if resp.status == 401:
return Unauthenticated(resp.status, code, detail)
if resp.status == 403 and code == "AGENT_ENV_MISMATCH":
return EnvMismatch(resp.status, code, detail)
if resp.status == 429:
try:
ra = int(resp.headers.get("retry-after", "60"))
except ValueError:
ra = 60
return RateLimited(resp.status, code, detail, ra)
return TenancyError(resp.status, code, detail)
class TenancyClient:
def __init__(self, http: HttpClient, env_id: str, token: str):
self.http = http
self.env_id = env_id
self.token = token
def _headers(self) -> dict[str, str]:
return {"Authorization": f"Bearer {self.token}"}
def _call(self, method: str, path: str, json_body=None) -> HttpResponse:
resp = self.http.request(method, path, json_body=json_body, headers=self._headers())
if resp.status >= 400:
raise _error(resp)
return resp
def checkin(
self,
*,
agent_version: str,
applied_sha: str | None,
host: dict | None,
containers: list[dict] | None,
) -> Checkin:
body = {
"env_id": self.env_id,
"agent_version": agent_version,
"applied_sha": applied_sha,
"host": host,
"containers": containers,
}
js = self._call("POST", "/v1/agent/checkin", body).json() or {}
action = js.get("action")
if action not in ("apply", "none", "down"):
raise TenancyError(200, "BAD_CHECKIN", f"unknown action {action!r}")
v = js.get("vault") or {}
return Checkin(
env_id=js.get("env_id") or self.env_id,
action=action,
desired_sha=js.get("desired_sha"),
purge_volumes=bool(js.get("purge_volumes", False)),
bundle_url=js.get("bundle_url"),
checkin_interval_s=int(js.get("checkin_interval_s") or 60),
vault=Vault(addr=v.get("addr"), mount=v.get("mount"), prefix=v.get("prefix")),
)
def bundle(self, url: str) -> tuple[bytes, str | None]:
"""The tar + the sha tenancy says it is (X-Bundle-Sha / X-Bundle-Sha256)."""
resp = self._call("GET", url)
ctype = resp.headers.get("content-type", "")
if "tar" not in ctype:
raise TenancyError(resp.status, "BAD_BUNDLE", f"unexpected content-type {ctype!r}")
sha = resp.headers.get("x-bundle-sha256") or resp.headers.get("x-bundle-sha")
return resp.body, sha
def lease(self, reason: str = "apply") -> Lease:
js = self._call("POST", "/v1/agent/lease", {"env_id": self.env_id, "reason": reason}).json() or {}
if "login_jwt" not in js:
if "wrapping_token" in js or "role_id" in js:
raise LeaseShapeUnsupported(
200,
"LEASE_SHAPE",
"tenancy issued an AppRole lease (wrapping_token/role_id); monky-deployd "
"requires the jwt-tenancy deploy grant {login_jwt, ttl_s, mount, role} "
"(ADR-0028 amendment 2026-09-05)",
)
raise TenancyError(200, "LEASE_SHAPE", "lease response carries no login_jwt")
v = js.get("vault") or {}
return Lease(
login_jwt=str(js["login_jwt"]),
ttl_s=int(js.get("ttl_s") or 3600),
mount=str(js.get("mount") or "jwt-tenancy"),
role=str(js.get("role") or "see-env"),
vault=Vault(addr=v.get("addr"), mount=v.get("mount"), prefix=v.get("prefix")),
)
def report(
self,
*,
sha: str | None,
result: str,
log_tail: str | None,
containers: list[dict] | None = None,
detail: str | None = None,
) -> None:
assert result in ("applied", "failed", "down")
body = {
"env_id": self.env_id,
"sha": sha,
"result": result,
"log_tail": (log_tail or "")[-LOG_TAIL_MAX:] or None,
"containers": containers,
}
if detail:
body["detail"] = detail[:500]
self._call("POST", "/v1/agent/report", body)
+221
View File
@@ -0,0 +1,221 @@
"""How the agent reaches the mesh, and a tiny HTTP client on top of it.
Three transports, one interface (`connect(host, port) -> socket`):
* `sdk` — the OpenZiti Python SDK dials the ziti service by its intercept name with the
box's own host identity (no tun, no root). Coexists with `ziti-edge-tunnel run-host`.
* `proxy` — `monky-deployd-proxy.service` runs `ziti tunnel proxy … monky.tenancy.deploy:18443
openbao:18200` as user ziti; the agent talks to 127.0.0.1:<port>. TLS SNI and
certificate checks still use the real hostname.
* `system` — plain DNS/TCP, for a laptop whose tunneler runs in `run` mode (tun + DNS).
Every network failure surfaces as `TransportError` (exit 75: temporary, retry next tick)."""
from __future__ import annotations
import http.client
import json
import logging
import socket
import ssl
from dataclasses import dataclass
from monky_deployd import __version__
from monky_deployd.config import Config
log = logging.getLogger("monky-deployd.transport")
USER_AGENT = f"monky-deployd/{__version__}"
class TransportError(Exception):
"""A network-level failure: DNS, connect, TLS, timeout, reset. Retry next tick."""
class Transport:
name = "base"
def connect(self, host: str, port: int, timeout: float) -> socket.socket: # pragma: no cover
raise NotImplementedError
def describe(self) -> str:
return self.name
class SystemTransport(Transport):
name = "system"
def connect(self, host: str, port: int, timeout: float) -> socket.socket:
return socket.create_connection((host, port), timeout=timeout)
class ProxyTransport(Transport):
"""(host, port) -> 127.0.0.1:<proxy port>; anything unmapped is refused (no leaks)."""
name = "proxy"
def __init__(self, mapping: dict[tuple[str, int], tuple[str, int]]):
self.mapping = mapping
def connect(self, host: str, port: int, timeout: float) -> socket.socket:
try:
target = self.mapping[(host, port)]
except KeyError as exc:
raise TransportError(f"no proxy mapping for {host}:{port}") from exc
return socket.create_connection(target, timeout=timeout)
def describe(self) -> str:
return "proxy(" + ", ".join(f"{h}:{p}->{t[0]}:{t[1]}" for (h, p), t in self.mapping.items()) + ")"
class SdkTransport(Transport):
"""`import openziti` is deferred so the other transports work without the wheel."""
name = "sdk"
def __init__(self, identity_path: str):
self.identity_path = identity_path
self._ctx = None
def _load(self):
if self._ctx is not None:
return
try:
import openziti # type: ignore
except ImportError as exc: # pragma: no cover - exercised via a fake module in tests
raise TransportError(
"transport sdk: the openziti module is not installed in this venv; "
"use transport: proxy (monky-deployd-proxy.service) or system"
) from exc
try:
self._ctx = openziti.load(self.identity_path)
except Exception as exc:
raise TransportError(f"transport sdk: cannot load identity {self.identity_path}: {exc}") from exc
self._openziti = openziti
def connect(self, host: str, port: int, timeout: float) -> socket.socket:
self._load()
# monkeypatch() swaps socket.socket for the SDK's ZitiSocket for the duration of the
# block: an address that matches a ziti intercept is dialled over the mesh, anything
# else falls through to a plain socket (which the proxy transport would have refused —
# for sdk that is what we want: bao.cbs.tikali.net is an intercept, not public DNS).
try:
with self._openziti.monkeypatch():
return socket.create_connection((host, port), timeout=timeout)
except OSError as exc:
raise TransportError(f"transport sdk: dial {host}:{port} failed: {exc}") from exc
def describe(self) -> str:
return f"sdk(identity={self.identity_path})"
def build(cfg: Config) -> Transport:
if cfg.transport == "system":
return SystemTransport()
_, bao_host, bao_port = cfg.bao_url
if cfg.transport == "proxy":
def _addr(s: str) -> tuple[str, int]:
h, _, p = s.rpartition(":")
return h.strip("[]") or "127.0.0.1", int(p)
return ProxyTransport(
{
(cfg.tenancy.host, cfg.tenancy.port): _addr(cfg.tenancy.proxy_addr),
(bao_host, bao_port): _addr(cfg.bao.proxy_addr),
}
)
return SdkTransport(cfg.identity)
# --- HTTP ---------------------------------------------------------------------------------------
@dataclass
class HttpResponse:
status: int
headers: dict[str, str]
body: bytes
def json(self):
if not self.body:
return None
try:
return json.loads(self.body.decode())
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise TransportError(f"non-JSON response ({self.status})") from exc
def text(self) -> str:
return self.body.decode(errors="replace")
class _Conn(http.client.HTTPConnection):
"""http.client with the TCP connect delegated to a Transport (+ optional TLS with SNI)."""
def __init__(self, transport: Transport, host: str, port: int, timeout: float, ctx: ssl.SSLContext | None):
super().__init__(host, port, timeout=timeout)
self._transport = transport
self._ctx = ctx
def connect(self) -> None:
sock = self._transport.connect(self.host, self.port, self.timeout)
if self._ctx is not None:
sock = self._ctx.wrap_socket(sock, server_hostname=self.host)
self.sock = sock
class HttpClient:
def __init__(
self,
transport: Transport,
scheme: str,
host: str,
port: int,
*,
ca_bundle: str | None = None,
timeout: float = 30,
):
self.transport = transport
self.scheme = scheme
self.host = host
self.port = port
self.timeout = timeout
self.ctx: ssl.SSLContext | None = None
if scheme == "https":
self.ctx = ssl.create_default_context(cafile=ca_bundle) if ca_bundle else ssl.create_default_context()
self.ctx.minimum_version = ssl.TLSVersion.TLSv1_2
@property
def base(self) -> str:
return f"{self.scheme}://{self.host}:{self.port}"
def request(
self,
method: str,
path: str,
*,
json_body=None,
headers: dict[str, str] | None = None,
body: bytes | None = None,
) -> HttpResponse:
hdrs = {"User-Agent": USER_AGENT, "Accept": "application/json, application/x-tar;q=0.9, */*;q=0.1"}
if headers:
hdrs.update(headers)
if json_body is not None:
body = json.dumps(json_body, separators=(",", ":")).encode()
hdrs["Content-Type"] = "application/json"
conn = _Conn(self.transport, self.host, self.port, self.timeout, self.ctx)
try:
conn.request(method, path, body=body, headers=hdrs)
resp = conn.getresponse()
data = resp.read()
return HttpResponse(resp.status, {k.lower(): v for k, v in resp.getheaders()}, data)
except TransportError:
raise
except (OSError, http.client.HTTPException, ssl.SSLError) as exc:
raise TransportError(f"{method} {self.base}{path}: {exc.__class__.__name__}: {exc}") from exc
finally:
try:
conn.close()
except Exception: # pragma: no cover
pass
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
# /usr/bin/monky-deployd -> the venv shipped in the .deb
exec /opt/monky-deployd/venv/bin/python -m monky_deployd "$@"
+222
View File
@@ -0,0 +1,222 @@
#!/usr/bin/env bash
# monky-deployd installer — Ubuntu 26.04 (verified target).
#
# curl -fsSL https://gitea.cbs.tikali.net/mdella/monky-deployd/raw/branch/main/packaging/install.sh \
# | sudo bash -s -- --env env-qa-02 --site cbs [--transport sdk|proxy|system] [--version 0.1.0] \
# [--enrol-jwt /path/monky-host.env-qa-02.jwt] [--laptop] < bootstrap.jwt
#
# stdin (or --bootstrap-file): the ONE-TIME bootstrap deploy grant from the install kit
# (GET /v1/backends/{id}/agent/install). The enrolment JWT is read from --enrol-jwt or
# /etc/monky-deployd/enrol.jwt and is only needed when the host identity is not enrolled yet.
#
# What it does (idempotent):
# 1. apt: ziti-edge-tunnel (OpenZiti `jammy` suite) if absent, docker-compose-plugin, acl
# 2. downloads the pinned monky-deployd_<ver>_amd64.deb + .sha256 from the Gitea release, verifies, installs
# 3. enrols /opt/openziti/etc/identities/monky-host.<env>.json if absent (ziti-edge-tunnel enroll),
# chown ziti:ziti 0600, switches ziti-edge-tunnel.service to `run-host` via a drop-in
# 4. writes /etc/monky-deployd/config.yaml, ACLs so user monky-deployd can read the identity,
# the bootstrap grant 0600, (proxy transport: proxy.env + monky-deployd-proxy.service)
# 5. enables monky-deployd.timer, runs one tick, deletes the enrol JWT, prints the checklist
set -euo pipefail
umask 077
DEFAULT_VERSION="0.1.0"
BASE_URL="${MONKY_DEPLOYD_BASE_URL:-https://gitea.cbs.tikali.net/mdella/monky-deployd}"
OPENZITI_SUITE="${OPENZITI_SUITE:-jammy}"
IDENTITY_DIR="/opt/openziti/etc/identities"
ETC="/etc/monky-deployd"
ENV_ID="" SITE="" TRANSPORT="sdk" VERSION="$DEFAULT_VERSION" ENROL_JWT="" BOOTSTRAP_FILE="" NO_RUN="" FORCE_CONFIG="" BAO_CA="" LAPTOP="false"
usage() { sed -n '2,20p' "$0"; exit "${1:-0}"; }
die() { echo "install.sh: $*" >&2; exit 1; }
log() { echo "==> $*"; }
while [ $# -gt 0 ]; do
case "$1" in
--env) ENV_ID="$2"; shift 2 ;;
--site) SITE="$2"; shift 2 ;;
--transport) TRANSPORT="$2"; shift 2 ;;
--version) VERSION="$2"; shift 2 ;;
--enrol-jwt) ENROL_JWT="$2"; shift 2 ;;
--bootstrap-file) BOOTSTRAP_FILE="$2"; shift 2 ;;
--bao-ca) BAO_CA="$2"; shift 2 ;;
--base-url) BASE_URL="$2"; shift 2 ;;
--laptop) LAPTOP="true"; shift ;;
--no-run) NO_RUN=1; shift ;;
--force-config) FORCE_CONFIG=1; shift ;;
-h|--help) usage 0 ;;
*) die "unknown argument $1 (see --help)" ;;
esac
done
[ "$(id -u)" = 0 ] || die "run as root (sudo)"
[ -n "$ENV_ID" ] || die "--env is required"
[ -n "$SITE" ] || die "--site is required"
[[ "$ENV_ID" =~ ^env-(dev|qa|stage|prod)-[0-9]{2,3}$|^(dev-env-2|prod-cedar)$ ]] || die "env id $ENV_ID is not env-<tier>-<nn>"
SITE="${SITE,,}"
[[ "$SITE" =~ ^(cbs|pdx)$ ]] || die "site must be cbs|pdx"
[[ "$TRANSPORT" =~ ^(sdk|proxy|system)$ ]] || die "transport must be sdk|proxy|system"
IDENTITY="$IDENTITY_DIR/monky-host.$ENV_ID.json"
export DEBIAN_FRONTEND=noninteractive
if [ -r /etc/os-release ]; then
. /etc/os-release
if [ "${ID:-}" != "ubuntu" ] || [ "${VERSION_ID:-}" != "26.04" ]; then
echo "WARNING: verified on Ubuntu 26.04; this is ${PRETTY_NAME:-unknown}. Continuing." >&2
fi
fi
# --- 1. packages -------------------------------------------------------------------------------------
apt_updated=""
apt_update_once() { [ -n "$apt_updated" ] || { apt-get update -qq; apt_updated=1; }; }
need_pkgs=(acl curl ca-certificates gnupg)
if ! command -v ziti-edge-tunnel >/dev/null 2>&1; then
log "adding the OpenZiti apt repository ($OPENZITI_SUITE suite)"
install -d -m 0755 /usr/share/keyrings
curl -fsSL https://get.openziti.io/tun/package-repos.gpg | gpg --dearmor -o /usr/share/keyrings/openziti.gpg
chmod 0644 /usr/share/keyrings/openziti.gpg
echo "deb [signed-by=/usr/share/keyrings/openziti.gpg] https://packages.openziti.org/zitipax-openziti-deb-stable $OPENZITI_SUITE main" \
> /etc/apt/sources.list.d/openziti.list
need_pkgs+=(ziti-edge-tunnel)
fi
if [ "$TRANSPORT" = "proxy" ] && ! command -v ziti >/dev/null 2>&1; then
need_pkgs+=(openziti) # the `ziti` CLI (ziti tunnel proxy) from the same repo
fi
command -v docker >/dev/null 2>&1 || die "docker is not installed; install Docker Engine first (https://docs.docker.com/engine/install/ubuntu/)"
docker compose version >/dev/null 2>&1 || need_pkgs+=(docker-compose-plugin)
python3 -c 'import sys; sys.exit(0 if sys.version_info >= (3, 12) else 1)' 2>/dev/null || die "python3 >= 3.12 is required"
if [ "${#need_pkgs[@]}" -gt 0 ]; then
log "apt install: ${need_pkgs[*]}"
apt_update_once
apt-get install -y -qq --no-install-recommends "${need_pkgs[@]}"
fi
# --- 2. the pinned .deb -----------------------------------------------------------------------------
installed="$(dpkg-query -W -f='${Version}' monky-deployd 2>/dev/null || true)"
if [ "$installed" = "$VERSION" ]; then
log "monky-deployd $VERSION already installed"
else
tmp="$(mktemp -d)"; trap 'rm -rf "$tmp"' EXIT
deb="monky-deployd_${VERSION}_amd64.deb"
url="$BASE_URL/releases/download/v${VERSION}"
log "downloading $deb from $url"
curl -fsSL -o "$tmp/$deb" "$url/$deb"
curl -fsSL -o "$tmp/$deb.sha256" "$url/$deb.sha256"
(cd "$tmp" && sha256sum -c "$deb.sha256") || die "sha256 mismatch on $deb"
apt_update_once || true
apt-get install -y -qq "$tmp/$deb"
fi
command -v monky-deployd >/dev/null || die "monky-deployd not on PATH after install"
# --- 3. host identity -------------------------------------------------------------------------------
install -d -m 0750 "$IDENTITY_DIR"
[ -n "$ENROL_JWT" ] || { [ -f "$ETC/enrol.jwt" ] && ENROL_JWT="$ETC/enrol.jwt"; } || true
if [ -s "$IDENTITY" ]; then
log "host identity present: $IDENTITY"
else
[ -n "$ENROL_JWT" ] && [ -s "$ENROL_JWT" ] || die "no identity at $IDENTITY and no enrol JWT (--enrol-jwt or $ETC/enrol.jwt)"
log "enrolling monky-host.$ENV_ID"
ziti-edge-tunnel enroll -j "$ENROL_JWT" -i "$IDENTITY"
fi
getent passwd ziti >/dev/null && chown ziti:ziti "$IDENTITY" || true
chmod 0600 "$IDENTITY"
# ziti-edge-tunnel as a HOST (bind side, no tun/DNS) — the agent dials with the SDK or the proxy
install -d /etc/systemd/system/ziti-edge-tunnel.service.d
cat > /etc/systemd/system/ziti-edge-tunnel.service.d/run-host.conf <<'DROPIN'
# monky-deployd: run-host mode (no tun, no DNS); identities from the standard directory
[Service]
ExecStart=
ExecStart=/opt/openziti/bin/ziti-edge-tunnel run-host --identity-dir=/opt/openziti/etc/identities
DROPIN
systemctl daemon-reload
systemctl enable --now ziti-edge-tunnel.service
systemctl restart ziti-edge-tunnel.service || true
# --- 4. config, ACLs, bootstrap grant -----------------------------------------------------------------
install -d -m 0750 -o root -g monky-deployd "$ETC"
install -d -m 0700 -o monky-deployd -g monky-deployd /var/lib/monky-deployd
setfacl -m u:monky-deployd:r "$IDENTITY"
setfacl -m u:monky-deployd:rx "$IDENTITY_DIR"
setfacl -m u:monky-deployd:x /opt/openziti/etc 2>/dev/null || true
if [ -n "$BAO_CA" ]; then
install -m 0644 "$BAO_CA" "$ETC/openbao-ca.pem"
fi
ca_line="ca_bundle: $ETC/openbao-ca.pem"
[ -s "$ETC/openbao-ca.pem" ] || { ca_line="ca_bundle: none # TODO: install the openbao-ca certificate (see docs/OPERATIONS.md)"; echo "WARNING: $ETC/openbao-ca.pem missing; TLS to OpenBao will use the system store" >&2; }
if [ -s "$ETC/config.yaml" ] && [ -z "$FORCE_CONFIG" ]; then
log "keeping existing $ETC/config.yaml (use --force-config to rewrite)"
else
cat > "$ETC/config.yaml" <<CFG
# written by install.sh $(date -u +%FT%TZ) — monky-deployd $VERSION
env_id: $ENV_ID
site: $SITE
transport: $TRANSPORT
identity: $IDENTITY
tenancy:
service: monky.tenancy.deploy
host: monky.tenancy.deploy
port: 8081
scheme: http
proxy_addr: 127.0.0.1:18443
bao:
service: openbao
addr: https://bao.cbs.tikali.net:8200
proxy_addr: 127.0.0.1:18200
$ca_line
mount: jwt-tenancy
role: see-env
kv_mount: monky
state_dir: /var/lib/monky-deployd
deploy_dir: /var/lib/monky-deployd/$ENV_ID
bootstrap_path: $ETC/bootstrap.jwt
interval_s: 60
volumes_on_absent: keep
laptop_mode: $LAPTOP
CFG
chmod 0640 "$ETC/config.yaml"; chgrp monky-deployd "$ETC/config.yaml"
fi
if [ "$TRANSPORT" = "proxy" ]; then
printf 'ZITI_IDENTITY=%s\n' "$IDENTITY" > "$ETC/proxy.env"; chmod 0644 "$ETC/proxy.env"
systemctl enable --now monky-deployd-proxy.service
fi
# bootstrap grant: stdin (the kit pipes it) or --bootstrap-file; 0600, owned by the agent so it can consume it
grant=""
if [ -n "$BOOTSTRAP_FILE" ]; then grant="$(tr -d '[:space:]' < "$BOOTSTRAP_FILE")"
elif [ ! -t 0 ]; then grant="$(tr -d '[:space:]' </dev/stdin || true)"; fi
if [ -n "$grant" ]; then
printf '%s\n' "$grant" > "$ETC/bootstrap.jwt"
chown monky-deployd:monky-deployd "$ETC/bootstrap.jwt"; chmod 0600 "$ETC/bootstrap.jwt"
log "bootstrap grant staged at $ETC/bootstrap.jwt (consumed on first tick)"
elif [ -s /var/lib/monky-deployd/bao.token ]; then
log "no bootstrap grant given; existing bao.token kept"
else
echo "WARNING: no bootstrap grant on stdin and no bao.token: the agent cannot check in until you provide one" >&2
fi
unset grant
# --- 5. timer + first tick ----------------------------------------------------------------------------------
systemctl daemon-reload
systemctl enable --now monky-deployd.timer
if [ -z "$NO_RUN" ]; then
log "first tick"
systemctl start monky-deployd.service || true
fi
[ -n "$ENROL_JWT" ] && rm -f "$ENROL_JWT" || true
rm -f "$ETC/enrol.jwt"
cat <<CHECK
monky-deployd $VERSION installed for $ENV_ID ($SITE, transport $TRANSPORT). Verification checklist (Ubuntu 26.04):
python3 --version # >= 3.12
docker compose version # compose plugin present
systemctl is-active ziti-edge-tunnel # run-host mode (drop-in run-host.conf)
$( [ "$TRANSPORT" = proxy ] && echo "systemctl is-active monky-deployd-proxy # ziti tunnel proxy on 18443/18200" || echo "ziti tunnel proxy --help >/dev/null # fallback transport available" )
/opt/monky-deployd/venv/bin/python -c 'import openziti' # SDK import (transport sdk)
monky-deployd status # token present, applied == desired
journalctl -u monky-deployd -n 50 # 'checkin:' and 'applied' lines
df -h \$(docker info -f '{{.DockerRootDir}}') # free space >= bundle need x 1.5 + headroom
docker compose -p monky-$ENV_ID ps # healthy
# from another mesh member: curl monky.percept.$ENV_ID:47283/health -> 200
CHECK
CHECK_STATUS="$(monky-deployd status 2>&1 || true)"
echo "$CHECK_STATUS" | sed 's/^/ | /'
+2
View File
@@ -0,0 +1,2 @@
# systemd-sysusers: the agent's service account (docker group membership is added by postinstall)
u monky-deployd - "Monky backend pull agent" /var/lib/monky-deployd
+2
View File
@@ -0,0 +1,2 @@
d /var/lib/monky-deployd 0700 monky-deployd monky-deployd -
d /etc/monky-deployd 0750 root monky-deployd -
+53
View File
@@ -0,0 +1,53 @@
# nfpm (https://nfpm.goreleaser.com) — builds monky-deployd_<ver>_amd64.deb in CI (`package` job).
# The venv under /opt/monky-deployd/venv is created by CI on ubuntu:26.04 with the system
# python3 (3.14 on 26.04; the agent needs >= 3.12) and the vendored openziti wheel when present.
name: monky-deployd
arch: amd64
platform: linux
version: ${VERSION}
section: admin
priority: optional
maintainer: Tikali <mdella@tikali.ai>
description: |
Monky backend pull agent (MONKY-ADR-0028): checkin -> bundle -> lease -> OpenBao -> docker compose -> report,
over the OpenZiti mesh with the box's own host identity.
vendor: Tikali
homepage: https://scm.tikali.ai/tikali/applications/monky/monky-deployd
license: Proprietary
depends:
- python3 (>= 3.12)
- acl
- ca-certificates
recommends:
- docker-compose-plugin
- ziti-edge-tunnel
contents:
- src: ./build/venv
dst: /opt/monky-deployd/venv
- src: ./packaging/bin/monky-deployd
dst: /usr/bin/monky-deployd
file_info: { mode: 0755 }
- src: ./packaging/systemd/monky-deployd.service
dst: /usr/lib/systemd/system/monky-deployd.service
- src: ./packaging/systemd/monky-deployd.timer
dst: /usr/lib/systemd/system/monky-deployd.timer
- src: ./packaging/systemd/monky-deployd-proxy.service
dst: /usr/lib/systemd/system/monky-deployd-proxy.service
- src: ./packaging/monky-deployd.sysusers
dst: /usr/lib/sysusers.d/monky-deployd.conf
- src: ./packaging/monky-deployd.tmpfiles
dst: /usr/lib/tmpfiles.d/monky-deployd.conf
- src: ./config.example.yaml
dst: /etc/monky-deployd/config.example.yaml
type: config
- src: ./packaging/install.sh
dst: /usr/share/monky-deployd/install.sh
file_info: { mode: 0755 }
- src: ./README.md
dst: /usr/share/doc/monky-deployd/README.md
- src: ./docs/OPERATIONS.md
dst: /usr/share/doc/monky-deployd/OPERATIONS.md
scripts:
postinstall: ./packaging/scripts/postinstall.sh
preremove: ./packaging/scripts/preremove.sh
postremove: ./packaging/scripts/postremove.sh
+22
View File
@@ -0,0 +1,22 @@
#!/bin/sh
set -e
# service account + dirs
if command -v systemd-sysusers >/dev/null 2>&1; then
systemd-sysusers /usr/lib/sysusers.d/monky-deployd.conf || true
else
getent passwd monky-deployd >/dev/null || useradd --system --home-dir /var/lib/monky-deployd --shell /usr/sbin/nologin monky-deployd
fi
if command -v systemd-tmpfiles >/dev/null 2>&1; then
systemd-tmpfiles --create /usr/lib/tmpfiles.d/monky-deployd.conf || true
fi
install -d -m 0700 -o monky-deployd -g monky-deployd /var/lib/monky-deployd
install -d -m 0750 -o root -g monky-deployd /etc/monky-deployd
# the agent drives docker compose: docker group membership (no root)
if getent group docker >/dev/null; then usermod -a -G docker monky-deployd || true; fi
# the venv is relocatable only to the path it was built at; refuse a broken interpreter early
/opt/monky-deployd/venv/bin/python -c 'import monky_deployd' || { echo "monky-deployd: venv unusable (python3 mismatch?)" >&2; exit 1; }
if [ -d /run/systemd/system ]; then
systemctl daemon-reload || true
# do NOT enable the timer here: install.sh / the ansible role do it after the config exists
fi
exit 0
+6
View File
@@ -0,0 +1,6 @@
#!/bin/sh
set -e
if [ -d /run/systemd/system ]; then systemctl daemon-reload || true; fi
# purge (dpkg -P) removes state; a plain remove keeps /var/lib/monky-deployd (token + releases)
if [ "$1" = "purge" ]; then rm -rf /var/lib/monky-deployd /etc/monky-deployd; fi
exit 0
+7
View File
@@ -0,0 +1,7 @@
#!/bin/sh
set -e
if [ -d /run/systemd/system ]; then
systemctl disable --now monky-deployd.timer 2>/dev/null || true
systemctl disable --now monky-deployd-proxy.service 2>/dev/null || true
fi
exit 0
@@ -0,0 +1,42 @@
# Fallback transport (b): `ziti tunnel proxy` publishes the two mesh services on loopback so the
# agent (transport: proxy) can reach them without the Python SDK. Runs as user ziti with the
# SAME host identity ziti-edge-tunnel run-host uses. Enable only with `transport: proxy`.
# monky.tenancy.deploy -> 127.0.0.1:18443 (plain HTTP inside the mesh)
# openbao -> 127.0.0.1:18200 (TLS end-to-end; SNI bao.cbs.tikali.net)
# NOTE: `ziti tunnel proxy` listens on IPv4 0.0.0.0 only (see project_openbao_consumer_gateway);
# the agent dials 127.0.0.1, so that is fine here.
[Unit]
Description=ziti tunnel proxy for monky-deployd (transport: proxy)
Documentation=https://scm.tikali.ai/tikali/applications/monky/monky-deployd
After=network-online.target
Wants=network-online.target
ConditionPathExists=/etc/monky-deployd/proxy.env
[Service]
Type=simple
User=ziti
Group=ziti
# proxy.env sets ZITI_IDENTITY=/opt/openziti/etc/identities/monky-host.<env>.json
EnvironmentFile=/etc/monky-deployd/proxy.env
ExecStart=/usr/bin/ziti tunnel proxy -i ${ZITI_IDENTITY} monky.tenancy.deploy:18443 openbao:18200
Restart=always
RestartSec=5s
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectControlGroups=yes
RestrictSUIDSGID=yes
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
CapabilityBoundingSet=
ReadOnlyPaths=-/opt/openziti/etc/identities
IPAddressDeny=any
IPAddressAllow=localhost
# the mesh edge routers are dialled by the ziti library itself: allow egress everywhere but
# only accept on loopback (the listeners bind 0.0.0.0 — IPAddressAllow above limits who connects)
IPAddressAllow=0.0.0.0/0 ::/0
[Install]
WantedBy=multi-user.target
+55
View File
@@ -0,0 +1,55 @@
# monky-deployd — one reconcile tick (MONKY-ADR-0028 §D). Fired by monky-deployd.timer.
[Unit]
Description=Monky backend pull agent (one tick: checkin -> bundle -> lease -> OpenBao -> compose -> report)
Documentation=https://scm.tikali.ai/tikali/applications/monky/monky-deployd
After=network-online.target docker.service ziti-edge-tunnel.service
Wants=network-online.target
# The proxy transport needs the local ziti proxy; harmless when the unit is disabled.
After=monky-deployd-proxy.service
ConditionPathExists=/etc/monky-deployd/config.yaml
[Service]
Type=oneshot
User=monky-deployd
Group=monky-deployd
SupplementaryGroups=docker
ExecStart=/usr/bin/monky-deployd run --once
# 75 = temporary network failure (next tick retries), 78 = AGENT_ENV_MISMATCH (operator action)
SuccessExitStatus=75
TimeoutStartSec=20min
Nice=5
Environment=PYTHONUNBUFFERED=1
Environment=MONKY_DEPLOYD_CONFIG=/etc/monky-deployd/config.yaml
UMask=0077
# hardening
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectControlGroups=yes
ProtectClock=yes
ProtectHostname=yes
RestrictSUIDSGID=yes
RestrictRealtime=yes
LockPersonality=yes
# (no MemoryDenyWriteExecute: the openziti SDK's ctypes callbacks need libffi trampolines)
RestrictNamespaces=yes
SystemCallArchitectures=native
SystemCallFilter=@system-service
SystemCallFilter=~@privileged @resources
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
CapabilityBoundingSet=
AmbientCapabilities=
StateDirectory=monky-deployd
StateDirectoryMode=0700
ReadWritePaths=/var/lib/monky-deployd
# the bootstrap grant is deleted after the first login (the only write under /etc)
ReadWritePaths=/etc/monky-deployd
ReadOnlyPaths=-/opt/openziti/etc/identities
# docker socket + compose need the socket path writable
ReadWritePaths=-/var/run/docker.sock -/run/docker.sock
[Install]
WantedBy=multi-user.target
+14
View File
@@ -0,0 +1,14 @@
# Every 60 s after the previous tick finished (+ up to 10 s jitter); flock in the agent prevents overlap.
[Unit]
Description=Monky backend pull agent tick timer
Documentation=https://scm.tikali.ai/tikali/applications/monky/monky-deployd
[Timer]
OnBootSec=90s
OnUnitActiveSec=60s
RandomizedDelaySec=10s
AccuracySec=1s
Unit=monky-deployd.service
[Install]
WantedBy=timers.target
+39
View File
@@ -0,0 +1,39 @@
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[project]
name = "monky-deployd"
version = "0.1.0"
description = "Monky backend pull agent: checkin -> bundle -> lease -> OpenBao -> docker compose -> report, over the ziti mesh (MONKY-ADR-0028)"
readme = "README.md"
requires-python = ">=3.12"
license = { text = "Proprietary" }
authors = [{ name = "Tikali", email = "mdella@tikali.ai" }]
dependencies = [] # stdlib only; `openziti` is optional (transport: sdk) and vendored by CI
[project.optional-dependencies]
sdk = ["openziti>=1.0"]
dev = ["pytest>=8", "ruff>=0.16,<0.17"]
[project.scripts]
monky-deployd = "monky_deployd.cli:main"
[tool.setuptools.packages.find]
include = ["monky_deployd*"]
[tool.ruff]
line-length = 120
target-version = "py312"
extend-exclude = ["ansible", "packaging"]
[tool.ruff.lint]
select = ["E", "F", "I", "UP", "B", "W"]
ignore = ["E501"]
[tool.ruff.lint.per-file-ignores]
"tests/*" = ["B011"]
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-q"
View File
+113
View File
@@ -0,0 +1,113 @@
from __future__ import annotations
import json
import os
import sys
from pathlib import Path
import pytest
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))
from monky_deployd import config as configmod # noqa: E402
from monky_deployd.redact import REDACTOR # noqa: E402
from tests.fakes import ENV, FakeBao, FakeTenancy, serve # noqa: E402
FAKEBIN = ROOT / "tests" / "fakebin"
@pytest.fixture
def bao():
b = FakeBao()
srv, port = serve(b)
b.port = port
yield b
srv.shutdown()
@pytest.fixture
def tenancy(bao):
t = FakeTenancy(bao)
srv, port = serve(t)
t.port = port
t.server = srv
yield t
try:
srv.shutdown()
except Exception:
pass
@pytest.fixture
def fake_docker(tmp_path, monkeypatch):
log = tmp_path / "docker.log"
state = tmp_path / "docker-state.json"
state.write_text(
json.dumps(
{
"ps": [{"Name": "see", "State": "running", "Health": "healthy"}],
"free_root": str(tmp_path),
"env_file_check": True,
}
)
)
monkeypatch.setenv("PATH", f"{FAKEBIN}:{os.environ['PATH']}")
monkeypatch.setenv("FAKE_DOCKER_LOG", str(log))
monkeypatch.setenv("FAKE_DOCKER_STATE", str(state))
class FD:
def calls(self):
if not log.exists():
return []
return [json.loads(line)["argv"] for line in log.read_text().splitlines()]
def subcommands(self):
out = []
for argv in self.calls():
if argv[:1] == ["compose"]:
rest = argv[1:]
while rest and rest[0].startswith("--"):
rest = rest[2:]
out.append("compose " + (rest[0] if rest else ""))
else:
out.append(" ".join(argv[:2]))
return out
def set(self, **kw):
cur = json.loads(state.read_text())
cur.update(kw)
state.write_text(json.dumps(cur))
return FD()
@pytest.fixture
def cfg(tmp_path, tenancy, bao, fake_docker):
REDACTOR.forget_all()
state_dir = tmp_path / "state"
etc = tmp_path / "etc"
etc.mkdir()
c = configmod.from_dict(
{
"env_id": ENV,
"site": "cbs",
"transport": "system",
"tenancy": {"host": "127.0.0.1", "port": tenancy.port, "scheme": "http"},
"bao": {"addr": f"http://127.0.0.1:{bao.port}", "ca_bundle": None},
"state_dir": str(state_dir),
"bootstrap_path": str(etc / "bootstrap.jwt"),
"interval_s": 60,
"healthy_timeout_s": 3,
"disk": {"factor": 1.5, "headroom_bytes": 1024},
},
path=str(etc / "config.yaml"),
)
return c
@pytest.fixture
def bootstrapped(cfg, bao):
"""A box with the kit's bootstrap grant on disk (first tick)."""
Path(cfg.bootstrap_path).write_text(bao.grant() + "\n")
return cfg
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/env python3
"""Fake `docker` for the hermetic tests. Records every invocation to $FAKE_DOCKER_LOG (one JSON
line per call) and answers from $FAKE_DOCKER_STATE (a JSON file the test writes):
{"ps": [{"Name": "see", "State": "running", "Health": "healthy"}], # compose ps output
"fail": ["pull"], # sub-commands that must exit 1
"free_root": "/tmp"} # DockerRootDir for `docker info`
"""
import json
import os
import sys
def main() -> int:
argv = sys.argv[1:]
state = {}
sp = os.environ.get("FAKE_DOCKER_STATE")
if sp and os.path.exists(sp):
with open(sp) as fh:
state = json.load(fh)
lp = os.environ.get("FAKE_DOCKER_LOG")
if lp:
with open(lp, "a") as fh:
fh.write(json.dumps({"argv": argv, "cwd": os.getcwd()}) + "\n")
fail = set(state.get("fail", []))
if argv[:1] == ["version"]:
print("28.3.0")
return 0
if argv[:1] == ["info"]:
print(state.get("free_root", "/"))
return 0
if argv[:2] == ["image", "prune"]:
return 0
if argv[:1] == ["compose"]:
# strip global compose flags
rest = argv[1:]
while rest and rest[0].startswith("--"):
rest = rest[2:]
sub = rest[0] if rest else ""
if sub == "version":
print("2.32.0")
return 0
if sub in fail:
print(f"fake docker: {sub} failed", file=sys.stderr)
return 1
if sub == "ps":
for row in state.get("ps", []):
print(json.dumps(row))
return 0
if sub == "logs":
print("fake compose logs")
return 0
if sub == "up" and state.get("env_file_check"):
# prove the .env is complete and 0600
d = os.getcwd()
env = os.path.join(d, ".env")
st = os.stat(env)
if st.st_mode & 0o077:
print("fake docker: .env is not 0600", file=sys.stderr)
return 1
return 0
print(f"fake docker: unknown {argv}", file=sys.stderr)
return 2
if __name__ == "__main__":
sys.exit(main())
+374
View File
@@ -0,0 +1,374 @@
"""Fake monky-tenancy agent endpoint + fake OpenBao, as real HTTP servers on 127.0.0.1.
They implement exactly the shapes of record (ADR-0028 + Gate 1 v2): the lease is a
`{login_jwt, ttl_s, mount, role, vault}` deploy grant; OpenBao's `auth/jwt-tenancy/login`
accepts any grant listed in `bao.grants` and pins the token's meta to that grant's env_id."""
from __future__ import annotations
import io
import json
import tarfile
import threading
import time
import uuid
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from monky_deployd.bundle import bundle_sha
ENV = "env-qa-02"
COMPOSE = """services:
see:
image: harbor.tikali.net/monky/see-backend:${SEE_TAG:-develop}
environment:
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
GEMINI_API_KEY: ${GEMINI_API_KEY}
SEE_ADMIN_TOKEN: ${SEE_ADMIN_TOKEN}
healthcheck:
test: ["CMD", "true"]
"""
def make_manifest(env=ENV, versions=None, prefix="monky"):
versions = versions or {}
entries = [
{
"var": "GEMINI_API_KEY",
"path": f"{prefix}/{env}/see/gemini_api_key",
"kind": "supplied",
"version": versions.get("gemini_api_key"),
},
{
"var": "POSTGRES_PASSWORD",
"path": f"{prefix}/{env}/see/pg_password",
"kind": "generated",
"version": versions.get("pg_password", 1),
},
{
"var": "SEE_ADMIN_TOKEN",
"path": f"{prefix}/{env}/see/admin_token",
"kind": "generated",
"version": versions.get("admin_token", 1),
},
]
return {"vault_mode": "openbao", "entries": entries}
def env_template(manifest):
lines = ["# .env template — PLACEHOLDERS ONLY", ""]
for e in manifest["entries"]:
lines += [f"# [{e['kind']}] openbao: {e['path']}", f"{e['var']}=${{{e['var']}}}"]
return "\n".join(lines) + "\n"
def make_files(env=ENV, *, compose=COMPOSE, manifest=None, meta=None):
manifest = manifest or make_manifest(env)
files = {
"docker-compose.yml": compose,
".env.template": env_template(manifest),
"secrets.manifest.json": json.dumps(manifest, indent=2, sort_keys=True) + "\n",
}
bundle_json = {
"env_id": env,
"tier": env.split("-")[1] if env.startswith("env-") else "dev",
"site": "cbs",
"bundle": "see",
"target_class": "docker-host",
"renderer": "compose",
"images_policy": "develop",
"secrets_provider": "openbao",
"agent": {},
"files": sorted([*files, "bundle.json"]),
}
if meta:
bundle_json.update(meta)
files["bundle.json"] = json.dumps(bundle_json, indent=2, sort_keys=True) + "\n"
return files
def tar_bytes(files: dict[str, str]) -> bytes:
buf = io.BytesIO()
with tarfile.open(fileobj=buf, mode="w") as tar:
for name in sorted(files):
data = files[name].encode()
info = tarfile.TarInfo(name=name)
info.size = len(data)
info.mode = 0o644
tar.addfile(info, io.BytesIO(data))
return buf.getvalue()
class _Server(ThreadingHTTPServer):
daemon_threads = True
allow_reuse_address = True
class _Handler(BaseHTTPRequestHandler):
server_version = "fake/0"
protocol_version = "HTTP/1.1"
def log_message(self, *a): # quiet
pass
def _body(self):
n = int(self.headers.get("Content-Length") or 0)
raw = self.rfile.read(n) if n else b""
return json.loads(raw) if raw else {}
def _send(self, status, body=None, *, raw=None, headers=None, ctype="application/json"):
data = raw if raw is not None else (json.dumps(body).encode() if body is not None else b"")
self.send_response(status)
self.send_header("Content-Type", ctype)
self.send_header("Content-Length", str(len(data)))
for k, v in (headers or {}).items():
self.send_header(k, v)
self.end_headers()
self.wfile.write(data)
def do_GET(self):
self.server.app.handle(self, "GET", self.path, {})
def do_POST(self):
self.server.app.handle(self, "POST", self.path, self._body())
class FakeBao:
"""auth/jwt-tenancy/login, token lookup-self/renew-self/revoke-self, KV-v2 reads."""
def __init__(self, env=ENV):
self.env = env
self.grants: dict[str, str] = {} # jwt -> env_id
self.tokens: dict[str, dict] = {}
self.kv: dict[str, list[dict]] = {} # "<env>/see/<name>" -> versions [ {value} ]
self.calls: list[tuple] = []
self.login_fail = False
self.token_ttl = 86400
self.max_ttl = 30 * 86400
self.mount = "jwt-tenancy"
self.role = "see-env"
self.kv_mount = "monky"
self.seed_secrets(env)
def seed_secrets(self, env):
self.kv[f"{env}/see/gemini_api_key"] = [
{"value": "AIzaSy-FAKE-GEMINI-KEY-0001"},
{"value": "AIzaSy-FAKE-GEMINI-KEY-0002"},
]
self.kv[f"{env}/see/pg_password"] = [{"value": 'pg-s3cret-"quoted"$dollar'}]
self.kv[f"{env}/see/admin_token"] = [{"value": "adm1n-t0ken-abcdef0123456789"}]
def grant(self, env=None, jti=None) -> str:
jwt = "eyJhbGciOiJFUzI1NiJ9." + uuid.uuid4().hex + uuid.uuid4().hex + "." + uuid.uuid4().hex + uuid.uuid4().hex
self.grants[jwt] = json.dumps({"env_id": env or self.env, "jti": jti or uuid.uuid4().hex})
return jwt
def mint(self, env=None, *, age_s=0, ttl=None) -> str:
tok = "hvs." + uuid.uuid4().hex + uuid.uuid4().hex[:8]
self.tokens[tok] = {
"accessor": "acc-" + uuid.uuid4().hex[:12],
"env_id": env or self.env,
"grant_jti": uuid.uuid4().hex,
"creation_time": int(time.time()) - age_s,
"ttl": ttl if ttl is not None else self.token_ttl,
"renewable": True,
"revoked": False,
}
return tok
def _auth(self, h):
tok = h.headers.get("X-Vault-Token")
t = self.tokens.get(tok)
if not t or t["revoked"]:
return None, tok
return t, tok
def handle(self, h, method, path, body):
self.calls.append((method, path.split("?")[0]))
if method == "POST" and path == f"/v1/auth/{self.mount}/login":
if self.login_fail:
return h._send(400, {"errors": ["error validating token: expired"]})
if body.get("role") != self.role:
return h._send(400, {"errors": [f"role {body.get('role')!r} could not be found"]})
meta = self.grants.pop(body.get("jwt", ""), None) # single-use grant
if meta is None:
return h._send(400, {"errors": ["error validating token: unknown or already used grant"]})
m = json.loads(meta)
tok = self.mint(m["env_id"])
self.tokens[tok]["grant_jti"] = m["jti"]
t = self.tokens[tok]
return h._send(
200,
{
"auth": {
"client_token": tok,
"accessor": t["accessor"],
"lease_duration": t["ttl"],
"renewable": True,
"token_policies": ["see-env"],
"metadata": {"env_id": m["env_id"], "grant_jti": m["jti"], "role": self.role},
}
},
)
t, tok = self._auth(h)
if t is None:
return h._send(403, {"errors": ["permission denied"]})
if method == "GET" and path == "/v1/auth/token/lookup-self":
return h._send(
200,
{
"data": {
"accessor": t["accessor"],
"creation_time": t["creation_time"],
"ttl": t["ttl"],
"renewable": t["renewable"],
"explicit_max_ttl": self.max_ttl,
"meta": {"env_id": t["env_id"], "grant_jti": t["grant_jti"]},
}
},
)
if method == "POST" and path == "/v1/auth/token/renew-self":
age = int(time.time()) - t["creation_time"]
t["ttl"] = max(0, min(self.token_ttl, self.max_ttl - age))
return h._send(200, {"auth": {"client_token": tok, "lease_duration": t["ttl"], "renewable": True}})
if method == "POST" and path == "/v1/auth/token/revoke-self":
t["revoked"] = True
return h._send(204)
if method == "GET" and path.startswith(f"/v1/{self.kv_mount}/data/"):
p, _, q = path.partition("?")
key = p[len(f"/v1/{self.kv_mount}/data/") :]
env = key.split("/")[0]
if env != t["env_id"]:
return h._send(403, {"errors": ["1 error occurred:\n\t* permission denied\n\n"]})
versions = self.kv.get(key)
if not versions:
return h._send(404, {"errors": []})
want = None
for part in q.split("&"):
if part.startswith("version="):
want = int(part.split("=", 1)[1])
idx = (want or len(versions)) - 1
if idx < 0 or idx >= len(versions):
return h._send(404, {"errors": []})
return h._send(200, {"data": {"data": versions[idx], "metadata": {"version": idx + 1}}})
return h._send(404, {"errors": [f"no handler for {method} {path}"]})
class FakeTenancy:
"""The agent entrypoint. `bao` validates bearers (a token for another env -> AGENT_ENV_MISMATCH)."""
def __init__(self, bao: FakeBao, env=ENV):
self.bao = bao
self.env = env
self.files = make_files(env)
self.desired_sha = bundle_sha(self.files)
self.action = "apply"
self.purge_volumes = False
self.checkins: list[dict] = []
self.reports: list[dict] = []
self.leases: list[dict] = []
self.lease_shape = "jwt" # or "approle" to simulate the un-migrated tenancy
self.lease_limit = 5
self.superseded_jtis: set[str] = set()
self.down = False # network down -> connection refused simulated by test stopping server
self.kv_mount = "monky"
def set_files(self, files):
self.files = files
self.desired_sha = bundle_sha(files)
def _principal(self, h):
auth = h.headers.get("Authorization", "")
tok = auth.split(" ", 1)[1] if auth.lower().startswith("bearer ") else None
t = self.bao.tokens.get(tok or "")
if not t or t["revoked"]:
return None, ("AGENT_UNAUTHENTICATED", "token unknown, expired or revoked")
if t["grant_jti"] in self.superseded_jtis:
return None, ("AGENT_UNAUTHENTICATED", "grant superseded")
return t, None
def _err(self, h, status, code, detail, headers=None):
return h._send(status, {"code": code, "detail": detail}, headers=headers)
def handle(self, h, method, path, body):
t, err = self._principal(h)
if err:
return self._err(h, 401, *err)
env_in = body.get("env_id") if body else None
if path.startswith("/v1/agent/bundle/"):
env_in = path.split("/")[4]
if env_in and env_in != t["env_id"]:
return self._err(h, 403, "AGENT_ENV_MISMATCH", f"token is pinned to {t['env_id']}, not {env_in}")
if method == "POST" and path == "/v1/agent/checkin":
self.checkins.append(body)
action = self.action
if action == "apply" and body.get("applied_sha") == self.desired_sha:
action = "none"
return h._send(
200,
{
"env_id": self.env,
"desired_sha": None if self.action == "down" else self.desired_sha,
"action": action,
"purge_volumes": self.purge_volumes if action == "down" else False,
"bundle_url": f"/v1/agent/bundle/{self.env}/{self.desired_sha}",
"checkin_interval_s": 60,
"vault": {
"addr": "https://bao.cbs.tikali.net:8200",
"mount": self.kv_mount,
"prefix": f"{self.env}/see",
},
},
)
if method == "GET" and path.startswith("/v1/agent/bundle/"):
sha = path.rsplit("/", 1)[1]
if sha != self.desired_sha:
return self._err(h, 404, "BUNDLE_NOT_FOUND", f"no published bundle {sha}")
return h._send(
200,
raw=tar_bytes(self.files),
ctype="application/x-tar",
headers={"Cache-Control": "no-store", "X-Bundle-Sha": sha},
)
if method == "POST" and path == "/v1/agent/lease":
if len(self.leases) >= self.lease_limit:
return self._err(h, 429, "LEASE_RATE_LIMITED", "5 leases per hour", headers={"Retry-After": "600"})
self.leases.append(body)
if self.lease_shape == "approle":
return h._send(
200,
{
"env_id": self.env,
"wrapping_token": "hvs.wrap-legacy",
"wrap_ttl_s": 3600,
"role_id": "role-id-see-env",
"vault": {},
},
)
jwt = self.bao.grant(self.env)
return h._send(
200,
{
"env_id": self.env,
"login_jwt": jwt,
"ttl_s": 3600,
"mount": self.bao.mount,
"role": self.bao.role,
"vault": {"addr": "https://bao.cbs.tikali.net:8200", "mount": self.kv_mount},
},
headers={"Cache-Control": "no-store"},
)
if method == "POST" and path == "/v1/agent/report":
self.reports.append(body)
if body.get("result") not in ("applied", "failed", "down"):
return self._err(h, 400, "BAD_REQUEST", "unknown result")
return h._send(200, {"env_id": self.env, "accepted": True, "result": body["result"]})
return self._err(h, 404, "NOT_FOUND", path)
def serve(app):
srv = _Server(("127.0.0.1", 0), _Handler)
srv.app = app
th = threading.Thread(target=srv.serve_forever, daemon=True)
th.start()
return srv, srv.server_address[1]
+321
View File
@@ -0,0 +1,321 @@
"""End-to-end ticks against the fake tenancy + fake OpenBao + fake docker."""
from __future__ import annotations
import json
import os
import time
from pathlib import Path
from monky_deployd import state as statemod
from monky_deployd.agent import EX_ENV_MISMATCH, EX_FAIL, EX_OK, EX_TEMPFAIL, Agent
from monky_deployd.bundle import bundle_sha
from tests.fakes import ENV, make_files, make_manifest
def tick(cfg, **kw):
return Agent(cfg, **kw).run_once()
def test_first_tick_bootstraps_applies_and_reports(bootstrapped, tenancy, bao, fake_docker):
cfg = bootstrapped
assert tick(cfg) == EX_OK
# bootstrap grant consumed, token persisted 0600
assert not Path(cfg.bootstrap_path).exists()
assert cfg.token_path.exists() and (cfg.token_path.stat().st_mode & 0o777) == 0o600
# protocol: checkin -> bundle -> lease -> report applied
assert tenancy.checkins[0]["env_id"] == ENV and tenancy.checkins[0]["agent_version"] == "0.1.0"
assert tenancy.checkins[0]["host"]["docker"] == "28.3.0"
assert len(tenancy.leases) == 1 and tenancy.leases[0]["reason"] == "apply"
assert [r["result"] for r in tenancy.reports] == ["applied"]
assert tenancy.reports[0]["sha"] == tenancy.desired_sha
assert tenancy.reports[0]["containers"] == [{"name": "see", "state": "running", "health": "healthy"}]
# OpenBao: login twice (bootstrap + lease), three KV reads pinned to versions
kv_calls = [p for m, p in bao.calls if "/data/" in p]
assert len(kv_calls) == 3 and all(f"/data/{ENV}/see/" in p for p in kv_calls)
# the lease token became the bearer; the bootstrap token was revoked
tokens = list(bao.tokens.values())
assert sum(1 for t in tokens if t["revoked"]) == 1
# files: .env 0600, complete, values never in logs; release promoted
current = Path(cfg.deploy_dir) / "current"
env = (current / ".env").read_text()
assert (current / ".env").stat().st_mode & 0o077 == 0
assert "GEMINI_API_KEY=AIzaSy-FAKE-GEMINI-KEY-0002" in env # latest version (manifest version None)
assert 'POSTGRES_PASSWORD="pg-s3cret-\\"quoted\\"$$dollar"' in env
assert "${" not in env.replace("${", "") or "=${" not in env
assert (current / "SHA256").read_text().strip() == tenancy.desired_sha
# compose: pull then up then ps
subs = fake_docker.subcommands()
assert "compose pull" in subs and "compose up" in subs and subs.index("compose pull") < subs.index("compose up")
# state
st = statemod.load(cfg.state_path, ENV)
assert st.applied_sha == tenancy.desired_sha and st.last_result == "applied" and st.token.source == "lease"
# log tail never carries a value
tail = tenancy.reports[0]["log_tail"]
assert tail and "AIzaSy" not in tail and "pg-s3cret" not in tail and "hvs." not in tail
def test_second_tick_is_a_heartbeat(bootstrapped, tenancy, fake_docker):
cfg = bootstrapped
assert tick(cfg) == EX_OK
assert tick(cfg) == EX_OK
assert [r["result"] for r in tenancy.reports] == ["applied", "applied"]
assert len(tenancy.leases) == 1 # no lease for a heartbeat
assert tenancy.checkins[1]["applied_sha"] == tenancy.desired_sha
assert fake_docker.subcommands().count("compose pull") == 1
def test_none_but_unhealthy_reports_failed(bootstrapped, tenancy, fake_docker):
cfg = bootstrapped
assert tick(cfg) == EX_OK
fake_docker.set(ps=[{"Name": "see", "State": "exited", "ExitCode": 137, "Health": ""}])
assert tick(cfg) == EX_FAIL
assert tenancy.reports[-1]["result"] == "failed" and "see=exited" in tenancy.reports[-1]["detail"]
def test_rotation_changes_sha_and_reapplies(bootstrapped, tenancy, bao, fake_docker):
cfg = bootstrapped
assert tick(cfg) == EX_OK
old = tenancy.desired_sha
tenancy.set_files(make_files(manifest=make_manifest(versions={"gemini_api_key": 1})))
assert tenancy.desired_sha != old
assert tick(cfg) == EX_OK
env = (Path(cfg.deploy_dir) / "current" / ".env").read_text()
assert "GEMINI_API_KEY=AIzaSy-FAKE-GEMINI-KEY-0001" in env # pinned version 1
st = statemod.load(cfg.state_path, ENV)
assert st.history == [old, tenancy.desired_sha]
assert len(list((Path(cfg.deploy_dir) / "releases").iterdir())) == 2
def test_prune_keeps_only_current(bootstrapped, tenancy, fake_docker):
cfg = bootstrapped
assert tick(cfg) == EX_OK
tenancy.set_files(make_files(manifest=make_manifest(versions={"gemini_api_key": 1})))
assert tick(cfg, prune=True) == EX_OK
assert [d.name for d in (Path(cfg.deploy_dir) / "releases").iterdir()] == [tenancy.desired_sha]
assert "image prune" in fake_docker.subcommands()
def test_rollback_refused_unless_allowed(bootstrapped, tenancy, fake_docker):
cfg = bootstrapped
first = make_files()
second = make_files(manifest=make_manifest(versions={"gemini_api_key": 1}))
assert tick(cfg) == EX_OK
tenancy.set_files(second)
assert tick(cfg) == EX_OK
tenancy.set_files(first)
assert tick(cfg) == EX_FAIL
assert tenancy.reports[-1]["result"] == "failed" and "ROLLBACK_REFUSED" in tenancy.reports[-1]["detail"]
assert len(tenancy.leases) == 2 # a refused bundle never leases
tenancy.set_files(make_files(meta={"agent": {"allow_rollback": True}}))
assert tick(cfg) == EX_OK
def test_down_removes_orphans_and_purges_only_when_allowed(bootstrapped, tenancy, fake_docker):
cfg = bootstrapped
assert tick(cfg) == EX_OK
tenancy.action, tenancy.purge_volumes = "down", True
assert tick(cfg) == EX_OK
down = [a for a in fake_docker.calls() if "down" in a][-1]
assert "--remove-orphans" in down and "-v" in down
assert tenancy.reports[-1] == {
"env_id": ENV,
"sha": None,
"result": "down",
"log_tail": tenancy.reports[-1]["log_tail"],
"containers": None,
}
assert statemod.load(cfg.state_path, ENV).applied_sha is None
assert not (Path(cfg.deploy_dir) / "current").exists()
def test_down_on_prod_never_purges(cfg, tenancy, bao, fake_docker):
cfg.prod = True
cfg.token_path.parent.mkdir(parents=True)
statemod.write_token(cfg.token_path, bao.mint())
tenancy.action, tenancy.purge_volumes = "down", True
assert tick(cfg) == EX_OK
down = [a for a in fake_docker.calls() if "down" in a][-1]
assert "-v" not in down and "--remove-orphans" in down
def test_env_mismatch_exits_78_and_writes_nothing(cfg, tenancy, bao, fake_docker):
cfg.token_path.parent.mkdir(parents=True)
statemod.write_token(cfg.token_path, bao.mint(env="env-dev-01"))
assert tick(cfg) == EX_ENV_MISMATCH
assert not (Path(cfg.deploy_dir) / "current").exists()
assert tenancy.reports == [] and tenancy.leases == []
assert "AGENT_ENV_MISMATCH" in statemod.load(cfg.state_path, ENV).last_error
def test_superseded_grant_drops_token_and_rebootstraps_if_grant_present(cfg, tenancy, bao, fake_docker):
cfg.token_path.parent.mkdir(parents=True)
old = bao.mint()
statemod.write_token(cfg.token_path, old)
tenancy.superseded_jtis.add(bao.tokens[old]["grant_jti"])
# no bootstrap grant -> exit 1, token removed, operator told to re-run the kit
assert tick(cfg) == EX_FAIL
assert not cfg.token_path.exists()
assert "AGENT_UNAUTHENTICATED" in statemod.load(cfg.state_path, ENV).last_error
# with a fresh kit grant on disk the same tick recovers
Path(cfg.bootstrap_path).write_text(bao.grant())
assert tick(cfg) == EX_OK
assert tenancy.reports[-1]["result"] == "applied"
def test_no_credentials_is_a_clear_exit_1(cfg, tenancy, fake_docker):
assert tick(cfg) == EX_FAIL
assert "install kit" in statemod.load(cfg.state_path, ENV).last_error
assert tenancy.checkins == []
def test_network_down_is_75_or_0_in_laptop_mode(cfg, tenancy, bao, fake_docker):
cfg.token_path.parent.mkdir(parents=True)
statemod.write_token(cfg.token_path, bao.mint())
tenancy.server.shutdown()
tenancy.server.server_close()
assert tick(cfg) == EX_TEMPFAIL
cfg.laptop_mode = True
assert tick(cfg) == EX_OK
assert "network" in statemod.load(cfg.state_path, ENV).last_error
def test_unresolved_var_is_env_incomplete_names_only(bootstrapped, tenancy, fake_docker):
cfg = bootstrapped
tenancy.set_files(
make_files(
compose="services:\n x:\n image: i\n environment:\n A: ${SOMETHING_MISSING}\n B: ${POSTGRES_PASSWORD}\n"
)
)
assert tick(cfg) == EX_FAIL
rep = tenancy.reports[-1]
assert rep["result"] == "failed" and "ENV_INCOMPLETE" in rep["detail"] and "SOMETHING_MISSING" in rep["detail"]
assert tenancy.leases == [] and "compose up" not in fake_docker.subcommands()
def test_privileged_refused_unless_bundle_allows(bootstrapped, tenancy, fake_docker):
cfg = bootstrapped
priv = "services:\n x:\n image: i\n privileged: true\n environment:\n B: ${POSTGRES_PASSWORD}\n A: ${GEMINI_API_KEY}\n C: ${SEE_ADMIN_TOKEN}\n"
tenancy.set_files(make_files(compose=priv))
assert tick(cfg) == EX_FAIL
assert "PRIVILEGED_REFUSED" in tenancy.reports[-1]["detail"]
tenancy.set_files(make_files(compose=priv, meta={"allow_privileged": True}))
assert tick(cfg) == EX_OK
def test_manifest_path_outside_env_refused_before_any_read(bootstrapped, tenancy, bao, fake_docker):
cfg = bootstrapped
tenancy.set_files(make_files(manifest=make_manifest(env="env-dev-01")))
assert tick(cfg) == EX_FAIL
assert "outside" in tenancy.reports[-1]["detail"]
assert not [p for m, p in bao.calls if "/data/" in p]
def test_bundle_env_mismatch_refused(bootstrapped, tenancy, fake_docker):
cfg = bootstrapped
tenancy.set_files(make_files(meta={"env_id": "env-dev-01"}))
assert tick(cfg) == EX_FAIL
assert "BUNDLE_ENV_MISMATCH" in tenancy.reports[-1]["detail"]
def test_sha_mismatch_refused(bootstrapped, tenancy, fake_docker):
cfg = bootstrapped
files = make_files()
tenancy.files = files
tenancy.desired_sha = "0" * 64 # tenancy claims a sha the tar does not hash to
assert tick(cfg) == EX_FAIL
assert "BUNDLE_SHA_MISMATCH" in tenancy.reports[-1]["detail"]
assert bundle_sha(files) != tenancy.desired_sha
def test_disk_refusal(bootstrapped, tenancy, fake_docker):
cfg = bootstrapped
tenancy.set_files(make_files(meta={"agent": {"disk_need_bytes": 10**18}}))
assert tick(cfg) == EX_FAIL
assert "DISK_INSUFFICIENT" in tenancy.reports[-1]["detail"]
assert tenancy.leases == []
def test_legacy_approle_lease_is_refused_loudly(bootstrapped, tenancy, fake_docker):
cfg = bootstrapped
tenancy.lease_shape = "approle"
assert tick(cfg) == EX_FAIL
rep = tenancy.reports[-1]
assert rep["result"] == "failed" and "LEASE_SHAPE" in rep["detail"] and "jwt-tenancy" in rep["detail"]
assert "compose up" not in fake_docker.subcommands()
def test_lease_rate_limited_is_temporary(bootstrapped, tenancy, fake_docker):
cfg = bootstrapped
tenancy.lease_limit = 0
assert tick(cfg) == EX_TEMPFAIL
def test_unhealthy_after_up_reports_failed_with_compose_logs(bootstrapped, tenancy, fake_docker):
cfg = bootstrapped
fake_docker.set(ps=[{"Name": "see", "State": "running", "Health": "starting"}])
assert tick(cfg) == EX_FAIL
rep = tenancy.reports[-1]
assert rep["result"] == "failed" and "fake compose logs" in rep["log_tail"]
assert statemod.load(cfg.state_path, ENV).applied_sha is None
def test_compose_pull_failure_reports_failed(bootstrapped, tenancy, fake_docker):
cfg = bootstrapped
fake_docker.set(fail=["pull"])
assert tick(cfg) == EX_FAIL
assert tenancy.reports[-1]["result"] == "failed" and "pull" in tenancy.reports[-1]["detail"]
def test_token_renew_and_release_before_max_ttl(bootstrapped, tenancy, bao, fake_docker):
cfg = bootstrapped
assert tick(cfg) == EX_OK
tok = cfg.token_path.read_text()
# low TTL, renewable -> renew-self
bao.tokens[tok]["ttl"] = 100
assert tick(cfg) == EX_OK
assert ("POST", "/v1/auth/token/renew-self") in bao.calls and bao.tokens[tok]["ttl"] == bao.token_ttl
# near max TTL -> re-lease (a new token replaces the old, which is revoked)
bao.tokens[tok]["creation_time"] = int(time.time()) - (bao.max_ttl - 3600)
st = statemod.load(cfg.state_path, ENV)
st.token.issued_at = time.time() - 7200
statemod.save(cfg.state_path, st)
assert tick(cfg) == EX_OK
assert tenancy.leases[-1]["reason"] == "renew"
new = cfg.token_path.read_text()
assert new != tok and bao.tokens[tok]["revoked"] is True and bao.tokens[new]["revoked"] is False
def test_dead_stored_token_is_dropped_by_upkeep(bootstrapped, tenancy, bao, fake_docker):
cfg = bootstrapped
assert tick(cfg) == EX_OK
tok = cfg.token_path.read_text()
bao.tokens[tok]["revoked"] = True
tenancy.action = "none"
# tenancy refuses the revoked bearer -> 401 path (no grant on disk) -> token dropped
assert tick(cfg) == EX_FAIL
assert not cfg.token_path.exists()
def test_lock_prevents_overlap(bootstrapped, fake_docker):
cfg = bootstrapped
lock = statemod.Lock(cfg.lock_path)
assert lock.acquire()
try:
assert tick(cfg) == EX_OK # skipped quietly
assert Path(cfg.bootstrap_path).exists() # nothing happened
finally:
lock.release()
def test_state_for_another_env_is_reset(tmp_path):
p = tmp_path / "state.json"
p.write_text(json.dumps({"env_id": "env-dev-01", "applied_sha": "x"}))
st = statemod.load(p, ENV)
assert st.env_id == ENV and st.applied_sha is None and "env-dev-01" in st.last_error
def test_write_private_mode(tmp_path):
p = tmp_path / "d" / "f"
statemod.write_private(p, b"x")
assert oct(p.stat().st_mode & 0o777) == "0o600" and not any(n.startswith(".f.") for n in os.listdir(p.parent))
+81
View File
@@ -0,0 +1,81 @@
import pytest
from monky_deployd import bundle as b
from monky_deployd.bao import ManifestPathError, kv_data_path
from tests.fakes import ENV, make_files, make_manifest, tar_bytes
def test_parse_and_sha_match_tenancy_formula():
files = make_files()
bd = b.parse(tar_bytes(files))
assert bd.sha == b.bundle_sha(files) and len(bd.sha) == 64
assert bd.compose_name == "docker-compose.yml"
assert bd.env_id == ENV and bd.tier == "qa"
assert b.manifest_vars(bd) == {"GEMINI_API_KEY", "POSTGRES_PASSWORD", "SEE_ADMIN_TOKEN"}
assert b.unresolved_vars(bd, b.manifest_vars(bd)) == []
def test_unresolved_vars_names_only_defaults_resolve():
files = make_files(
compose="services:\n x:\n image: i:${TAG:-dev}\n environment:\n A: ${NOT_IN_MANIFEST}\n B: ${POSTGRES_PASSWORD}\n"
)
bd = b.parse(tar_bytes(files))
assert b.unresolved_vars(bd, b.manifest_vars(bd)) == ["NOT_IN_MANIFEST"]
def test_privileged_and_host_network_detected():
files = make_files(
compose="services:\n x:\n image: i\n privileged: true\n network_mode: host\n cap_add:\n - SYS_ADMIN\n"
)
bd = b.parse(tar_bytes(files))
assert b.privileged_findings(bd) == ["privileged: true", "network_mode: host", "cap_add SYS_ADMIN/ALL"]
assert bd.flag("allow_privileged") is False
bd2 = b.parse(tar_bytes(make_files(meta={"agent": {"allow_privileged": True}})))
assert bd2.flag("allow_privileged") is True
def test_manifest_with_value_is_refused():
m = make_manifest()
m["entries"][0]["value"] = "leaked"
with pytest.raises(b.BundleError, match="carries a value"):
b.parse(tar_bytes(make_files(manifest=m)))
def test_unsafe_members_refused():
import io
import tarfile
buf = io.BytesIO()
with tarfile.open(fileobj=buf, mode="w") as tar:
info = tarfile.TarInfo(name="../etc/passwd")
info.size = 1
tar.addfile(info, io.BytesIO(b"x"))
with pytest.raises(b.BundleError, match="unsafe"):
b.parse(buf.getvalue())
def test_render_env_quotes_the_compose_way():
tmpl = "# c\nA=${A}\nB=${B}\nC=${C}\nKEEP=${KEEP}\n"
out = b.render_env(tmpl, {"A": "plain-value.1", "B": 'has "quote" and $dollar', "C": "multi\nline"})
assert 'A=plain-value.1\nB="has \\"quote\\" and $$dollar"\nC="multi\\nline"\nKEEP=${KEEP}\n' in out
assert b.referenced_vars(out) == {"KEEP"}
def test_kv_paths_are_pinned_to_the_env():
assert kv_data_path("monky", f"monky/{ENV}/see/pg_password", ENV) == f"/v1/monky/data/{ENV}/see/pg_password"
assert kv_data_path("monky", f"monky/data/{ENV}/see/x", ENV) == f"/v1/monky/data/{ENV}/see/x"
for bad in (
"monky/env-dev-01/see/x",
"monky/companies/c1/ai/gemini",
"other/env-qa-02/see/x",
f"monky/{ENV}/zitadel/x",
f"monky/{ENV}/see/../x",
):
with pytest.raises(ManifestPathError):
kv_data_path("monky", bad, ENV)
def test_disk_need_bytes_spellings():
assert b.parse(tar_bytes(make_files(meta={"agent": {"disk_need_bytes": 5}}))).disk_need_bytes == 5
assert b.parse(tar_bytes(make_files(meta={"disk": {"need_bytes": 7}}))).disk_need_bytes == 7
assert b.parse(tar_bytes(make_files())).disk_need_bytes == 0
+102
View File
@@ -0,0 +1,102 @@
import json
from pathlib import Path
from monky_deployd import cli
from monky_deployd.agent import EX_OK
def write_cfg(cfg):
lines = [
f"env_id: {cfg.env_id}",
f"site: {cfg.site}",
"transport: system",
"tenancy:",
f" host: {cfg.tenancy.host}",
f" port: {cfg.tenancy.port}",
" scheme: http",
"bao:",
f" addr: {cfg.bao.addr}",
" ca_bundle: none",
f"state_dir: {cfg.state_dir}",
f"bootstrap_path: {cfg.bootstrap_path}",
f"deploy_dir: {cfg.deploy_dir}",
"healthy_timeout_s: 3",
]
Path(cfg.path).write_text("\n".join(lines) + "\n")
return cfg.path
def test_run_once_then_status(bootstrapped, tenancy, fake_docker, capsys):
path = write_cfg(bootstrapped)
assert cli.main(["-c", path, "run", "--once"]) == EX_OK
assert cli.main(["-c", path, "status", "--json"]) == EX_OK
out = json.loads(capsys.readouterr().out)
assert out["in_sync"] is True and out["applied_sha"] == tenancy.desired_sha and out["token_present"] is True
assert out["healthy"] is True and out["containers"][0]["name"] == "see"
assert cli.main(["-c", path, "status"]) == EX_OK
text = capsys.readouterr().out
assert "in sync" in text and "healthy" in text and "hvs." not in text
def test_bootstrap_command(bootstrapped, bao, capsys):
cfg = bootstrapped
path = write_cfg(cfg)
assert cli.main(["-c", path, "bootstrap"]) == EX_OK
assert cfg.token_path.exists() and not Path(cfg.bootstrap_path).exists()
assert cli.main(["-c", path, "bootstrap"]) == EX_OK
assert "already bootstrapped" in capsys.readouterr().out
def test_version_and_bad_config(capsys, tmp_path):
assert cli.main(["version"]) == 0
assert capsys.readouterr().out.strip() == "0.1.0"
bad = tmp_path / "c.yaml"
bad.write_text("env_id: nope\nsite: cbs\n")
assert cli.main(["-c", str(bad), "status"]) == 78
assert cli.main(["-c", str(tmp_path / "missing.yaml"), "status"]) == 1
def test_sdk_transport_uses_openziti_monkeypatch(monkeypatch, tmp_path):
"""The sdk transport loads the identity once and dials inside openziti.monkeypatch()."""
import contextlib
import socket
import sys
import types
calls = []
fake = types.ModuleType("openziti")
fake.load = lambda p: calls.append(("load", p)) or object()
@contextlib.contextmanager
def mp():
calls.append(("monkeypatch",))
yield
fake.monkeypatch = mp
monkeypatch.setitem(sys.modules, "openziti", fake)
from monky_deployd.transport import SdkTransport
srv = socket.socket()
srv.bind(("127.0.0.1", 0))
srv.listen(1)
t = SdkTransport(str(tmp_path / "id.json"))
s = t.connect("127.0.0.1", srv.getsockname()[1], 2)
s.close()
srv.close()
assert calls == [("load", str(tmp_path / "id.json")), ("monkeypatch",)]
assert "sdk(identity=" in t.describe()
def test_proxy_transport_refuses_unmapped_hosts():
from monky_deployd.config import from_dict
from monky_deployd.transport import TransportError, build
cfg = from_dict({"env_id": "env-dev-06", "site": "cbs", "transport": "proxy"})
t = build(cfg)
assert t.mapping[("monky.tenancy.deploy", 8081)] == ("127.0.0.1", 18443)
assert t.mapping[("bao.cbs.tikali.net", 8200)] == ("127.0.0.1", 18200)
try:
t.connect("example.com", 443, 1)
raise AssertionError("unmapped host must be refused")
except TransportError:
pass
+79
View File
@@ -0,0 +1,79 @@
import pytest
from monky_deployd import config as c
KIT = """\
# written by the install kit
env_id: env-qa-02
site: cbs
transport: sdk
identity: /opt/openziti/etc/identities/monky-host.env-qa-02.json
tenancy:
service: monky.tenancy.deploy
base_url: http://monky.tenancy.deploy:8081
bao:
service: openbao
addr: https://bao.cbs.tikali.net:8200 # intercept, not public DNS
ca_bundle: /etc/monky-deployd/openbao-ca.pem
mount: jwt-tenancy
role: see-env
kv_mount: monky
interval_s: 60
laptop_mode: false
volumes_on_absent: keep
"""
def test_yaml_subset_parses_nested_maps_and_types():
d = c.parse_yaml_subset(KIT)
assert d["env_id"] == "env-qa-02"
assert d["tenancy"]["base_url"] == "http://monky.tenancy.deploy:8081"
assert d["bao"]["addr"] == "https://bao.cbs.tikali.net:8200"
assert d["interval_s"] == 60 and d["laptop_mode"] is False
def test_yaml_subset_lists_quotes_and_comments():
d = c.parse_yaml_subset("a: \"x # not a comment\"\nb: 'q'\nlist:\n - one\n - 2\nn: ~\n")
assert d == {"a": "x # not a comment", "b": "q", "list": ["one", 2], "n": None}
def test_yaml_subset_refuses_flow_style_and_tabs():
with pytest.raises(c.ConfigError):
c.parse_yaml_subset("a: [1, 2]\n")
with pytest.raises(c.ConfigError):
c.parse_yaml_subset("a:\n\tb: 1\n")
def test_config_defaults_and_derivations():
cfg = c.from_dict(c.parse_yaml_subset(KIT))
assert cfg.tenancy.host == "monky.tenancy.deploy" and cfg.tenancy.port == 8081 and cfg.tenancy.scheme == "http"
assert cfg.bao_url == ("https", "bao.cbs.tikali.net", 8200)
assert cfg.deploy_dir == "/var/lib/monky-deployd/env-qa-02"
assert cfg.compose_project == "monky-env-qa-02"
assert str(cfg.token_path) == "/var/lib/monky-deployd/bao.token"
assert cfg.is_prod is False
assert cfg.bao.mount == "jwt-tenancy" and cfg.bao.role == "see-env"
def test_config_prod_detection_and_legacy_ids():
assert c.from_dict({"env_id": "env-prod-01", "site": "pdx"}).is_prod is True
assert c.from_dict({"env_id": "prod-cedar", "site": "cbs"}).is_prod is True
assert c.from_dict({"env_id": "dev-env-2", "site": "cbs"}).is_prod is False
with pytest.raises(c.ConfigError):
c.from_dict({"env_id": "dev-env-1", "site": "cbs"}) # retired 2026-09-04
def test_config_rejects_approle_and_unknown_keys():
with pytest.raises(c.ConfigError, match="approle"):
c.from_dict({"env_id": "env-dev-06", "site": "cbs", "bao": {"approle": {"path": "approle"}}})
with pytest.raises(c.ConfigError, match="unknown key"):
c.from_dict({"env_id": "env-dev-06", "site": "cbs", "tenancy": {"nope": 1}})
with pytest.raises(c.ConfigError, match="transport"):
c.from_dict({"env_id": "env-dev-06", "site": "cbs", "transport": "carrier-pigeon"})
with pytest.raises(c.ConfigError, match="site"):
c.from_dict({"env_id": "env-dev-06", "site": "sfo"})
def test_sdk_identity_defaults_to_host_identity():
cfg = c.from_dict({"env_id": "env-dev-07", "site": "cbs"})
assert cfg.identity == "/opt/openziti/etc/identities/monky-host.env-dev-07.json"
+27
View File
@@ -0,0 +1,27 @@
import logging
from monky_deployd.redact import MASK, Redactor
def test_redacts_token_shapes_and_registered_values():
r = Redactor()
r.add("pg-s3cret-value")
text = "tok hvs.CAESIJabcdefghijklmnopqrstuvwxyz0123456789 jwt eyJhbGciOiJFUzI1NiJ9.eyJzdWIiOiJhZ2VudCJ9.c2lnbmF0dXJlLXNpZw pw=pg-s3cret-value ok"
out = r.scrub(text)
assert "hvs." not in out and "eyJ" not in out and "pg-s3cret-value" not in out
assert out.count(MASK) == 3
assert r.scrub("Authorization: Bearer abc.def") == f"Authorization: Bearer {MASK}"
def test_filter_rewrites_records_in_place():
r = Redactor()
r.add("SUPERSECRET")
rec = logging.LogRecord("x", logging.INFO, "f", 1, "value=%s", ("SUPERSECRET",), None)
assert r.filter(rec) is True
assert rec.getMessage() == f"value={MASK}"
def test_short_values_are_not_registered():
r = Redactor()
r.add("ab")
assert r.scrub("ab is fine") == "ab is fine"