15 Commits

Author SHA1 Message Date
mdella b749a6d3b4 Merge branch 'docs/dd-0524-0526-protocol-shapes' into 'main'
docs(protocol): AppRole divergence is resolved history; lease example + fake follow tenancy's AgentLeaseOut (DD-0524/DD-0526)

See merge request tikali/applications/monky/monky-deployd!8
2026-09-06 23:23:32 -07:00
Claude-Docs-Manager 05724edde2 docs(protocol): the AppRole divergence is resolved history; the lease example and fake follow tenancy's AgentLeaseOut
DD-0524 — PROTOCOL.md §Divergences bullets 1-2 said monky-tenancy `main`
"still implements the AppRole lease and install kit" and that "the kit's
generated config uses tenancy.base_url". Both were false when the page was
published: tenancy !17 (288df791, merged 07:48Z) shipped AgentLeaseOut{env_id,
login_jwt, ttl_s, mount, role, addr}, the ES256 grant signer and the JWKS
sixteen minutes before v0.1.0 was tagged, and !22 (61bd0281) made the kit run
install.sh with flags instead of writing a config. The two bullets are now
dated "Resolved" notes; bullets 3-4 (report `detail`, X-Bundle-Sha) stand.

DD-0526 — the lease example sent `reason` and received a nested `vault{}`;
tenancy's AgentLease is `{env_id}` and AgentLeaseOut carries `addr` at the top
level (no vault object). The example now shows tenancy's shapes (with a note
that the agent still sends `reason` and tenancy ignores unknown fields), the
checkin example gains `auth_mount`/`auth_role` so it is the full AgentVaultOut,
tests/fakes.py emits `addr` the way tenancy does, and tenancy.py's docstring
and lease() read `addr` first (the `vault{}` fallback is kept so an older fake
or tenancy still leases). Verified against monky-tenancy app/schemas_backends.py
at 1fd51454 (AgentLease 296-297, AgentLeaseOut 300-309, AgentVaultOut 278-283).

Gates (local, py3.12): ruff format --check, ruff check, pytest 50 passed,
bash -n packaging/install.sh.

Doc-Drift: DD-0524 fixed
Doc-Drift: DD-0526 fixed
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AW3QqEpwLV69KHn24Re45Q
2026-09-06 23:13:09 -07:00
monky-deployd-merger b39a60af46 Merge branch 'fix/tenancy-intercept-port' into 'main'
fix: tenancy.port is the intercept port (443); clear sdk dial error; 0.1.5

See merge request tikali/applications/monky/monky-deployd!7
2026-09-07 06:12:42 +00:00
mdella fcf2dcb1eb fix(bundle): ${VAR} inside comment lines is not a reference
The renderer's .env.template header literally says "substitutes every
${VAR}", which the refusal check counted as an unresolved variable
(ENV_INCOMPLETE: unresolved: VAR) — env-qa-02's first bundle was refused.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
2026-09-07 06:09:08 +00:00
mdella 31586c3058 fix: tenancy.port is the intercept port (443); clear sdk dial error; 0.1.5
install.sh wrote tenancy.port 8081 (the in-pod port) but the
monky.tenancy.deploy intercept is 443, so the SDK found no intercept and the
check-in failed with 'service not available' followed by the SDK's bare
TypeError. Also wrap non-OSError SDK failures in a TransportError that names
the host:port and what to check.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
2026-09-07 06:05:21 +00:00
monky-deployd-merger 41541629ec Merge branch 'fix/tunnel-execstartpre' into 'main'
fix(install.sh): clear ExecStartPre in the run-host drop-in; config/grant before aborting; 0.1.4

See merge request tikali/applications/monky/monky-deployd!6
2026-09-07 06:00:16 +00:00
mdella 744041ca1e fix(install.sh): clear ExecStartPre in the run-host drop-in; don't abort before config/grant land; 0.1.4
On env-qa-02 (ziti-edge-tunnel 1.18.7) the package's ExecStartPre script is
not executable by user ziti (203/EXEC), so the unit never started, and under
set -e `systemctl enable --now` aborted install.sh before config.yaml, ACLs,
the bootstrap grant and the timer were written — a second kit was needed.

- run-host drop-in clears ExecStartPre= (run-host does not auto-enrol JWTs)
- a tunneler start failure is a WARNING until everything else is installed,
  then a hard error at the end (re-run needs no new kit)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
2026-09-07 05:58:57 +00:00
monky-deployd-merger b08e0adf82 Merge branch 'fix/install-version-clobber-docker' into 'main'
fix(install.sh): --version clobbered by os-release; Docker Engine install; 0.1.3

See merge request tikali/applications/monky/monky-deployd!5
2026-09-06 16:20:12 +00:00
mdella 837a7e5bb1 fix(install.sh): --version clobbered by /etc/os-release; install Docker Engine when absent; 0.1.3
The first real kit run (env-qa-02) downloaded
monky-deployd_26.04 LTS (Resolute Raccoon)_amd64.deb: sourcing /etc/os-release
inline overwrote VERSION. Script vars are now DEPLOYD_VERSION and os-release is
read in a subshell.

Per the operator's rule (end users run ONE script), install.sh now installs
Docker Engine when absent — Docker's apt suite for the host codename, falling
back to Ubuntu's docker.io — with --docker-data-root <dir> to place the
data-root before first start. Existing Docker is left untouched.

bash -n + shellcheck -S warning clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
2026-09-06 16:18:47 +00:00
monky-deployd-merger f8a35947db Merge branch 'feat/deploy-token-download' into 'main'
feat(install): read-only deploy token for the private package registry (v0.1.2)

See merge request tikali/applications/monky/monky-deployd!4
2026-09-05 19:10:24 +00:00
mdella 6336012b74 feat(install): read-only deploy token for the private package registry (v0.1.2)
The GitLab project is private (its parent groups are private, so it cannot be
made public): the v0.1.1 one-liner answered 401 anonymously. install.sh gains
--token / MONKY_DEPLOYD_TOKEN and sends `DEPLOY-TOKEN: <token>` (a GitLab deploy
token, scope read_package_registry only, revocable) on every registry download,
the script itself included; the token goes through a 0600 curl -K file (never the
command line, the log or an xtrace). The grant is taken via --bootstrap-file when
the script is piped (stdin IS the script). Ansible: monky_deployd_download_token
(vaulted) -> DEPLOY-TOKEN header, no_log. Docs explain why, the token's scope and
the --source gitea alternative (split-horizon Gitea, cbs/iac#102).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
2026-09-05 19:03:25 +00:00
monky-deployd-merger bd3f8c1b6e Merge branch 'fix/install-urls-scm' into 'main'
fix(install): fetch from scm.tikali.ai (public project) — Gitea name is split-horizon inside the estate

See merge request tikali/applications/monky/monky-deployd!3
2026-09-05 17:59:27 +00:00
mdella c966450d8e fix(install): fetch from scm.tikali.ai (public project) — Gitea name is split-horizon inside the estate
Inside the estate gitea.cbs.tikali.net resolves to jump1's RED EIP (10.10.0.175),
which has no HTTP ingress, so backend boxes could not download the install
artefacts from the Gitea mirror (cbs/iac#102). scm.tikali.ai is reachable from
those boxes and the project is now public, so the GitLab generic package
registry becomes the PRIMARY source:

- packaging/install.sh: default source = scm.tikali.ai generic package registry
  (projects/69/packages/generic/monky-deployd/<ver>/...); `--source gitea` /
  MONKY_DEPLOYD_SOURCE=gitea keeps the Gitea release as the off-estate
  alternative; --base-url / MONKY_DEPLOYD_BASE_URL still override the base.
- ansible role defaults: monky_deployd_base_url/_deb_url point at the registry,
  Gitea layout kept as a commented alternative.
- README / docs/OPERATIONS.md / CLAUDE.md / CI comments + release description:
  both locations keep being published (release + release:gitea).
- Version 0.1.1 (the tag gate refuses v* tags whose version != __version__);
  tests compare against __version__ instead of a literal. No agent change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
2026-09-05 17:57:27 +00:00
monky-deployd-merger 28f7cf110f Merge branch 'ci/proven-egress' into 'main'
ci: wheel/package blocking — runner egress proven (v0.1.0 pipeline)

See merge request tikali/applications/monky/monky-deployd!2
2026-09-05 08:12:43 +00:00
mdella 8fde0ba079 ci: wheel/package are blocking — runner egress proven on the v0.1.0 tag pipeline
Pipeline 6999 built the openziti 1.7.1 wheel (github.com + pypi.org) and the
nfpm .deb; the allow_failure escape hatches are no longer honest. package now
requires the wheel so a .deb can never ship without transport sdk.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
2026-09-05 08:11:25 +00:00
20 changed files with 348 additions and 94 deletions
+20 -16
View File
@@ -3,12 +3,16 @@
# lint -> test -> build (openziti wheel) -> package (.deb via nfpm) -> release (v* tags) -> docs
#
# Every script line is single-quoted (a bare ": " turns the line into a YAML map and silently
# yields a 0-job pipeline). Jobs that need egress the runner may not have (GitHub for the
# openziti sdist's ziti-sdk-c fetch, GitHub for the nfpm binary) are allow_failure: true until
# proven; see README "CI notes".
# yields a 0-job pipeline). Runner egress to github.com + pypi.org was PROVEN on the v0.1.0 tag
# pipeline (6999: openziti 1.7.1 wheel built, nfpm .deb packaged), so wheel/package are blocking.
#
# CI/CD variables (project or group level):
# GITEA_TOKEN — Gitea API token (write:repository) for `release:gitea`; without it the job is manual
#
# Release assets are published to BOTH the GitLab generic package registry (the installer's primary
# source — the project is PRIVATE (parent groups are private), so install.sh sends a read-only deploy
# token (read_package_registry) as DEPLOY-TOKEN; gitea.cbs.tikali.net is split-horizon inside the
# estate, cbs/iac#102) and the Gitea mirror release (`install.sh --source gitea`, off-estate).
# GITLAB_TRANSLATE_TOKEN — used by the translate-docs component (group level)
include:
@@ -76,13 +80,11 @@ test:
# PyPI ships `openziti` as an sdist whose build fetches ziti-sdk-c (+ prebuilt tlsuv/uv-mbed
# via cmake FetchContent) from github.com at install time. Building it here on ubuntu:26.04
# (the target OS; python3 = the target's python3) gives us a wheel to vendor into the venv.
# NEEDS runner egress to github.com + pypi.org; allow_failure until proven on this runner —
# without the wheel the .deb still builds (transports proxy/system work; sdk logs a clear error).
# NEEDS runner egress to github.com + pypi.org (proven 2026-09-05, pipeline 6999).
wheel:
stage: build
image: ubuntu:26.04
needs: ["test"]
allow_failure: true
rules:
- if: '$CI_COMMIT_TAG'
- if: '$CI_COMMIT_BRANCH == "main"'
@@ -104,15 +106,14 @@ wheel:
# --- the .deb --------------------------------------------------------------------------------------
# venv at its final path (/opt/monky-deployd/venv is where the .deb puts it; venvs are not
# relocatable) + nfpm. nfpm comes from GitHub releases (egress) with the goreleaser apt repo as
# fallback; allow_failure until proven.
# fallback (proven 2026-09-05, pipeline 6999). The wheel is required: a .deb without it would
# silently ship a broken `transport: sdk`.
package:
stage: package
image: ubuntu:26.04
needs:
- job: test
- job: wheel
optional: true
allow_failure: true
rules:
- if: '$CI_COMMIT_TAG'
- if: '$CI_COMMIT_BRANCH == "main"'
@@ -127,7 +128,9 @@ package:
- 'python3 -m venv /opt/monky-deployd/venv'
- '/opt/monky-deployd/venv/bin/pip install -q --upgrade pip'
- '/opt/monky-deployd/venv/bin/pip install -q .'
- 'if ls vendor/*.whl >/dev/null 2>&1; then /opt/monky-deployd/venv/bin/pip install -q vendor/*.whl && /opt/monky-deployd/venv/bin/python -c "import openziti; print(\"openziti\", openziti.__version__ if hasattr(openziti, \"__version__\") else \"ok\")"; else echo "WARNING: no vendored openziti wheel — transport sdk will not work from this build"; fi'
- 'ls vendor/*.whl >/dev/null 2>&1 || { echo "no vendored openziti wheel (wheel job artifact missing)"; exit 1; }'
- '/opt/monky-deployd/venv/bin/pip install -q vendor/*.whl'
- '/opt/monky-deployd/venv/bin/python -c "import openziti; print(\"openziti import ok\")"'
- '/opt/monky-deployd/venv/bin/python -m monky_deployd version'
- 'mkdir -p build dist && cp -a /opt/monky-deployd/venv build/venv'
# nfpm: GitHub release .deb, else the goreleaser apt repo
@@ -141,7 +144,8 @@ package:
dotenv: build.env
expire_in: 90 days
# --- GitLab release (v* tags): generic package registry + release with asset links -------------
# --- GitLab release (v* tags): generic package registry (the PRIMARY download) + release ---------
# install.sh / the ansible role fetch ${CI_API_V4_URL}/projects/69/packages/generic/monky-deployd/<ver>/...
release:
stage: release
image:
@@ -157,15 +161,15 @@ release:
- 'DEB=$(basename dist/*.deb)'
- >-
release-cli create --name "monky-deployd $CI_COMMIT_TAG" --tag-name "$CI_COMMIT_TAG"
--description "See CHANGELOG.md. Public install assets are on the Gitea mirror: https://gitea.cbs.tikali.net/mdella/monky-deployd/releases/tag/$CI_COMMIT_TAG"
--description "See CHANGELOG.md. Install assets: this release's package registry links (primary; the install.sh default) and the Gitea mirror for off-estate use: https://gitea.cbs.tikali.net/mdella/monky-deployd/releases/tag/$CI_COMMIT_TAG"
--assets-link "{\"name\":\"$DEB\",\"url\":\"$PKG/$DEB\",\"link_type\":\"package\"}"
--assets-link "{\"name\":\"$DEB.sha256\",\"url\":\"$PKG/$DEB.sha256\",\"link_type\":\"other\"}"
--assets-link "{\"name\":\"install.sh\",\"url\":\"$PKG/install.sh\",\"link_type\":\"other\"}"
# --- Gitea release (the PUBLIC download the installer uses) ------------------------------------------
# The GitLab project is private, so install.sh fetches from the Gitea mirror
# https://gitea.cbs.tikali.net/mdella/monky-deployd/releases/download/v<ver>/... . This job
# waits for the pull-mirror to carry the tag, creates the release and uploads the assets.
# --- Gitea release (the off-estate alternative: install.sh --source gitea) -----------------------
# Same assets at https://gitea.cbs.tikali.net/mdella/monky-deployd/releases/download/v<ver>/... .
# Not the default: inside the estate that name is split-horizon to jump1's RED EIP (cbs/iac#102).
# This job waits for the pull-mirror to carry the tag, creates the release and uploads the assets.
# Automatic when GITEA_TOKEN is set; otherwise a manual, non-blocking job (docs/OPERATIONS.md
# has the by-hand recipe).
release:gitea:
+55
View File
@@ -1,6 +1,61 @@
<!-- xlate:verbatim-fences -->
# Changelog
## 0.1.5 — 2026-09-07
- **`${VAR}` inside comment lines is not a reference.** The renderer's `.env.template` header literally says
"substitutes every ${VAR}", which the refusal check counted as an unresolved variable
(`ENV_INCOMPLETE: unresolved: VAR`) — the first bundle on env-qa-02 was refused for it.
- **config: `tenancy.port` is the service's intercept port (443), not the in-pod 8081.** With 8081 the SDK
found no intercept and the dial failed (`service not available`, then a bare `TypeError` from the SDK's
fallback). `install.sh` now writes 443; `config.example.yaml` updated.
- **transport sdk: clear error instead of a TypeError** when an address has no intercept or the identity
has no dial policy for the service (`TransportError` names the host:port and what to check).
## 0.1.4 — 2026-09-07
- **install.sh: `ziti-edge-tunnel.service` failed to start after enrolment** (`203/EXEC`: the package's
`ExecStartPre` script is not executable by user `ziti` on ziti-edge-tunnel 1.18.x) and, because the
unit was enabled with `--now` under `set -e`, the script aborted BEFORE writing config/ACLs/grant/timer —
a second kit was needed. The run-host drop-in now clears `ExecStartPre=`; a tunneler start failure is a
WARNING until everything else is in place, then a hard error at the end.
## 0.1.3 — 2026-09-06
- **install.sh: `--version` was clobbered by `/etc/os-release`** (it defines `VERSION`), so the kit tried to download
`monky-deployd_26.04 LTS (Resolute Raccoon)_amd64.deb` (env-qa-02 pilot, first real kit run). Script variables are now
`DEPLOYD_VERSION`; os-release is read in a subshell.
- **install.sh installs Docker Engine when absent** (Docker's apt suite for the codename → fallback `docker.io`), with
`--docker-data-root <dir>` to place the data-root before first start. One script for the end user, per the operator's rule.
## v0.1.2 — 2026-09-05
- **The GitLab project is private** (its parent groups are private, so it cannot be made public;
found when the v0.1.1 one-liner returned 401 anonymously). `install.sh` gains `--token <deploy-token>`
/ `MONKY_DEPLOYD_TOKEN`: every download from the generic package registry — the script itself
included — sends `DEPLOY-TOKEN: <token>`, a read-only GitLab **deploy token** (scope
`read_package_registry` only, revocable), seeded in OpenBao at `monky/monky-tenancy/deployd-download`
(key `token`) and handed to the box by the monky-tenancy install kit. The token never reaches the
command line, the log or an xtrace (curl `-K` config file, 0600, deleted after the download;
`set +x` forced). The one-liner now fetches `install.sh` from the registry
(`…/monky-deployd/<ver>/install.sh`) instead of `-/raw/main`, and takes the bootstrap grant via
`--bootstrap-file` — with `curl … | bash -s --` stdin IS the script, so it is no longer read for
the grant in that mode. `--source gitea` (no token) stays the off-estate alternative.
- Ansible role: `monky_deployd_download_token` (vaulted) → `DEPLOY-TOKEN` header on both fetches,
`no_log: true`.
- README / OPERATIONS: why (private project; split-horizon Gitea, cbs/iac#102), token scope, revocation.
No agent behaviour change.
## v0.1.1 — 2026-09-05
- `install.sh` / ansible role / README: the **primary download is the public GitLab project's generic
package registry on scm.tikali.ai** (`/api/v4/projects/69/packages/generic/monky-deployd/<ver>/…`);
the one-liner fetches the script from `scm.tikali.ai/…/-/raw/main/packaging/install.sh`. Inside the
estate `gitea.cbs.tikali.net` is split-horizon to jump1's RED EIP (no HTTP ingress), so backend
boxes could not fetch the artefacts (cbs/iac#102). The Gitea release stays the off-estate
alternative (`--source gitea` / `MONKY_DEPLOYD_SOURCE=gitea`). No agent behaviour change.
## v0.1.0 — 2026-09-05
First release (MONKY-ADR-0028 §D, Reconciliation v2, Gate 1 v2).
+4 -2
View File
@@ -75,8 +75,10 @@ systemd-analyze verify packaging/systemd/*.service # when available
Bump `monky_deployd.__version__` + `pyproject.toml` + `packaging/install.sh` `DEFAULT_VERSION` +
`CHANGELOG.md`, merge to `main`, tag `vX.Y.Z` (protected `v*`). The tag pipeline builds the `.deb`,
publishes the GitLab release and — with `GITEA_TOKEN` — the **Gitea release** the installer
downloads from (`docs/OPERATIONS.md` has the manual recipe). `lint` refuses a tag whose version
publishes the **GitLab generic package registry + release** the installer downloads from (PRIVATE
project — the installer sends a read-only deploy token as `DEPLOY-TOKEN`; the Gitea name is
split-horizon inside the estate, cbs/iac#102) and — with `GITEA_TOKEN` —
the **Gitea release** (`install.sh --source gitea`, off-estate; `docs/OPERATIONS.md` has the manual recipe). `lint` refuses a tag whose version
differs from `__version__`.
## What NOT to do
+36 -11
View File
@@ -18,20 +18,37 @@ Verified on **Ubuntu 26.04**.
## Install (one-liner, from the enrolment kit)
An admin reveals the kit once in the console (`GET /v1/backends/{id}/agent/install`); it hands
you the enrolment JWT and a one-time **bootstrap deploy grant**. On the box:
you the enrolment JWT, a one-time **bootstrap deploy grant** and the read-only **download token**
(the kit runs the line below for you). By hand, on the box:
```sh
curl -fsSL https://gitea.cbs.tikali.net/mdella/monky-deployd/raw/branch/main/packaging/install.sh \
| sudo bash -s -- --env env-qa-02 --site cbs --enrol-jwt ./monky-host.env-qa-02.jwt < bootstrap.jwt
# [--transport sdk|proxy|system] [--version 0.1.0] [--laptop] [--bao-ca openbao-ca.pem]
T=<deploy token> # read-only GitLab deploy token (read_package_registry); the kit carries it
curl -sSf -H "DEPLOY-TOKEN: $T" https://scm.tikali.ai/api/v4/projects/69/packages/generic/monky-deployd/0.1.5/install.sh \
| sudo bash -s -- --env env-qa-02 --site cbs --token "$T" --bootstrap-file bootstrap.jwt --enrol-jwt ./monky-host.env-qa-02.jwt
# [--transport sdk|proxy|system] [--version 0.1.5] [--docker-data-root /home/docker-data] [--laptop] [--bao-ca openbao-ca.pem] [--source gitlab|gitea]
```
`install.sh` installs `ziti-edge-tunnel` (OpenZiti `jammy` suite) and `docker-compose-plugin` if
absent, downloads the pinned `.deb` + `.sha256` from the [Gitea release](https://gitea.cbs.tikali.net/mdella/monky-deployd/releases),
absent, downloads the pinned `.deb` + `.sha256` from the [scm.tikali.ai generic package registry](https://scm.tikali.ai/tikali/applications/monky/monky-deployd/-/packages)
(`https://scm.tikali.ai/api/v4/projects/69/packages/generic/monky-deployd/<ver>/monky-deployd_<ver>_amd64.deb`),
enrols `monky-host.<env>` if the identity is missing, switches the tunneler to `run-host`,
writes `/etc/monky-deployd/config.yaml`, grants the agent read access to the identity (ACL),
stages the bootstrap grant (0600), enables `monky-deployd.timer`, runs one tick and deletes the
JWT. The GitLab project is private, so **the public download is the Gitea mirror**.
JWT.
**Why a token.** The GitLab project is **private** and cannot be made public (its parent groups
are private), so anonymous downloads answer 401. Every registry fetch — the script itself
included — therefore sends `DEPLOY-TOKEN: <token>`: a GitLab **deploy token** with the single
scope `read_package_registry` (it can download packages and nothing else: no code, no API, no
write; revoke it in the project's *Settings → Repository → Deploy tokens* and issue a new one).
The operator seeds it in OpenBao at `monky/monky-tenancy/deployd-download` (key `token`); the
monky-tenancy install kit reads it from there and passes `--token` (`MONKY_DEPLOYD_TOKEN` also
works). The script never prints it (curl `-K` config file, 0600, deleted after the download;
`set -x` is switched off). Why not the Gitea mirror: inside the estate `gitea.cbs.tikali.net` is
split-horizon to jump1's RED EIP (`10.10.0.175`), which has no HTTP ingress, so backend boxes
cannot reach it (cbs/iac#102); `--source gitea` (or `MONKY_DEPLOYD_SOURCE=gitea`, no token) keeps
the [Gitea release](https://gitea.cbs.tikali.net/mdella/monky-deployd/releases) as the
off-estate alternative.
## Transports
@@ -138,14 +155,22 @@ systemd-analyze verify packaging/systemd/*.service # where systemd is availabl
`lint` and `test` run on every MR/branch. `wheel` builds the `openziti` wheel on `ubuntu:26.04`
(PyPI ships an sdist that fetches **ziti-sdk-c from github.com** at build time — the runner needs
egress to github.com and pypi.org) and `package` builds the `.deb` with `nfpm` (binary from GitHub
releases, goreleaser apt repo as fallback). Both are `allow_failure: true` until proven on this
runner; without the wheel the `.deb` still works with `transport: proxy|system`. On a `v*` tag
`release` uploads to the GitLab generic package registry + release, and `release:gitea` publishes
the same assets on the public Gitea mirror (automatic when `GITEA_TOKEN` is set, manual otherwise
— see `docs/OPERATIONS.md` for the by-hand recipe).
releases, goreleaser apt repo as fallback). Both were proven on the v0.1.0 tag pipeline and are
blocking on `main`/tags (manual on MRs); the `.deb` always ships the SDK wheel. On a `v*` tag
`release` uploads to the GitLab generic package registry + release (**the primary download**; the
project is private, so the installer sends the read-only deploy token), and `release:gitea` publishes the same assets on the Gitea mirror (the `--source gitea`
alternative; automatic when `GITEA_TOKEN` is set, manual otherwise — see `docs/OPERATIONS.md` for
the by-hand recipe). Both locations keep being published.
## See also
- `docs/PROTOCOL.md`, `docs/OPERATIONS.md`, `CHANGELOG.md`
- monky-tenancy `docs/usage.md` (agent protocol), `app/api/agent.py`, `app/schemas_backends.py`
- monky-deploy (the renderer whose `render_files` produces the bundle)
### Docker Engine (since 0.1.3)
`install.sh` installs Docker Engine when it is absent (Docker's apt suite for the host's Ubuntu codename, falling back to
Ubuntu's `docker.io`), so an end user runs exactly one script. `--docker-data-root <dir>` writes `/etc/docker/daemon.json`
before the daemon first starts (pilot VMs keep the data-root on a dedicated disk, e.g. `/home/docker-data`). A pre-existing
Docker is left untouched. macOS/Windows laptops still need Docker Desktop from the user.
+6 -2
View File
@@ -4,7 +4,9 @@ Installs and configures [monky-deployd](https://scm.tikali.ai/tikali/application
(the Monky backend pull agent, MONKY-ADR-0028) on a docker host that already carries an enrolled
host identity (`roles/ziti_tunneler`, run-host mode). Skeleton for **osg1-07**; copy it there.
What it does: pin + download the `.deb` from the Gitea release (sha256 verified) → ACL
What it does: pin + download the `.deb` from the scm.tikali.ai package registry (sha256 verified; the
project is **private**, so the fetches send the read-only deploy token `monky_deployd_download_token`
as `DEPLOY-TOKEN`, `no_log`; `monky_deployd_base_url`/`_deb_url` switch to the Gitea release off-estate) → ACL
`u:monky-deployd:r` on the identity (`rx` on the dir) → `/etc/monky-deployd/config.yaml` from the
template → optional openbao-ca PEM → optional `monky-deployd-proxy.service` (`transport: proxy`)
→ the **one-time bootstrap deploy grant** from a vault var (0600, `no_log`) → `monky-deployd.timer`
@@ -14,7 +16,8 @@ template → optional openbao-ca PEM → optional `monky-deployd-proxy.service`
| var | note |
|---|---|
| `monky_deployd_version` | pinned release, e.g. `0.1.0` |
| `monky_deployd_version` | pinned release, e.g. `0.1.5` |
| `monky_deployd_download_token` | **vaulted**: GitLab deploy token, scope `read_package_registry` only (revocable) — the registry is private; seeded in OpenBao at `monky/monky-tenancy/deployd-download` key `token` (path/key are the operator's choice). Empty = no header (only works with the Gitea `base_url`) |
| `monky_deployd_env_id` / `_site` / `_transport` | per host (`env-dev-06`, `cbs`, `sdk`) |
| `monky_deployd_bootstrap_grant` | tenancy-minted deploy grant (1 h) — `ansible-vault` or a lookup at play time; empty keeps the existing token |
| `monky_deployd_bao_ca_pem` | the `openbao-ca` certificate (PEM) |
@@ -30,6 +33,7 @@ template → optional openbao-ca PEM → optional `monky-deployd-proxy.service`
- role: monky_deployd
vars:
monky_deployd_bootstrap_grant: "{{ lookup('pipe', 'tenancy-mint-grant ' ~ inventory_hostname) }}"
monky_deployd_download_token: "{{ vault_monky_deployd_download_token }}" # ansible-vault / OpenBao lookup
```
Rollout order (plan §E): pilot env-qa-02 → env-dev-06..09 → env-dev-01 last (after its live key moves into Bao).
+16 -3
View File
@@ -1,11 +1,24 @@
---
# monky_deployd — install and configure the Monky backend pull agent (MONKY-ADR-0028 §D).
# Copy this role into osg1-07 (roles/monky_deployd) and roll to env-dev-06..09 after the pilot.
monky_deployd_version: "0.1.0"
monky_deployd_base_url: "https://gitea.cbs.tikali.net/mdella/monky-deployd"
monky_deployd_version: "0.1.5"
monky_deployd_deb: "monky-deployd_{{ monky_deployd_version }}_amd64.deb"
monky_deployd_deb_url: "{{ monky_deployd_base_url }}/releases/download/v{{ monky_deployd_version }}/{{ monky_deployd_deb }}"
# PRIMARY download = the GitLab project's generic package registry on scm.tikali.ai. Inside the estate
# gitea.cbs.tikali.net is split-horizon to jump1's RED EIP (10.10.0.175, no HTTP ingress), so backend
# boxes cannot fetch from the Gitea mirror (cbs/iac#102). The project is PRIVATE (its parent groups are
# private), so the registry needs a read-only GitLab deploy token (scope read_package_registry only,
# revocable) sent as the `DEPLOY-TOKEN` header — `monky_deployd_download_token`, a VAULTED var
# (ansible-vault or an OpenBao lookup: KV `monky/monky-tenancy/deployd-download`, key `token`).
# Off-estate alternative (Gitea release, no token):
# monky_deployd_base_url: "https://gitea.cbs.tikali.net/mdella/monky-deployd"
# monky_deployd_deb_url: "{{ monky_deployd_base_url }}/releases/download/v{{ monky_deployd_version }}/{{ monky_deployd_deb }}"
# monky_deployd_download_token: ""
monky_deployd_base_url: "https://scm.tikali.ai/api/v4/projects/69/packages/generic/monky-deployd"
monky_deployd_deb_url: "{{ monky_deployd_base_url }}/{{ monky_deployd_version }}/{{ monky_deployd_deb }}"
monky_deployd_deb_sha256_url: "{{ monky_deployd_deb_url }}.sha256"
monky_deployd_download_token: ""
# the header the download tasks send (empty map when no token)
monky_deployd_download_headers: "{{ {'DEPLOY-TOKEN': monky_deployd_download_token} if monky_deployd_download_token | length > 0 else {} }}"
# per host (inventory / host_vars)
monky_deployd_env_id: "{{ inventory_hostname }}" # env-dev-06 ...
+7 -1
View File
@@ -22,20 +22,26 @@
changed_when: false
failed_when: false
- name: monky_deployd | download .deb + sha256 from the Gitea release
# The registry is PRIVATE: both fetches send `DEPLOY-TOKEN: {{ monky_deployd_download_token }}`
# (read_package_registry only) and run no_log so the header never reaches the play output.
- name: monky_deployd | download .deb + sha256 from the scm.tikali.ai package registry
when: monky_deployd_installed.stdout != monky_deployd_version
block:
- name: monky_deployd | fetch sha256
ansible.builtin.uri:
url: "{{ monky_deployd_deb_sha256_url }}"
headers: "{{ monky_deployd_download_headers }}"
return_content: true
register: monky_deployd_sha
no_log: true
- name: monky_deployd | fetch .deb (checksum verified)
ansible.builtin.get_url:
url: "{{ monky_deployd_deb_url }}"
headers: "{{ monky_deployd_download_headers }}"
dest: "/var/cache/apt/archives/{{ monky_deployd_deb }}"
checksum: "sha256:{{ monky_deployd_sha.content.split()[0] }}"
mode: "0644"
no_log: true
- name: monky_deployd | install .deb
ansible.builtin.apt:
deb: "/var/cache/apt/archives/{{ monky_deployd_deb }}"
+2 -2
View File
@@ -1,4 +1,4 @@
# /etc/monky-deployd/config.yaml — monky-deployd v0.1.0 (MONKY-ADR-0028 §D)
# /etc/monky-deployd/config.yaml — monky-deployd v0.1.5 (MONKY-ADR-0028 §D)
# Written by packaging/install.sh (or the ansible role monky_deployd). YAML *subset*: maps, scalars,
# simple lists, comments. Keys not listed here are a config error.
@@ -10,7 +10,7 @@ identity: /opt/openziti/etc/identities/monky-host.env-qa-02.json # the box's h
tenancy:
service: monky.tenancy.deploy # ziti service bound by the tenancy sidecar -> 127.0.0.1:8081 (agent entrypoint)
host: monky.tenancy.deploy # intercept host (sdk/system); defaults to `service`
port: 8081
port: 443 # intercept port of monky.tenancy.deploy (in-pod 8081)
scheme: http # plain HTTP inside the mesh; the mesh is the transport security
proxy_addr: 127.0.0.1:18443 # transport: proxy
timeout_s: 30
+27 -3
View File
@@ -86,6 +86,30 @@ the picker, nothing is retired automatically.
`docker image prune -f` after a successful apply. The state dir and the token survive `apt remove`;
`apt purge` deletes them.
## Where the installer downloads from
The primary source is the **scm.tikali.ai generic package registry** of this project:
`https://scm.tikali.ai/api/v4/projects/69/packages/generic/monky-deployd/<ver>/<file>` for
`monky-deployd_<ver>_amd64.deb`, `.sha256` and `install.sh`. The project is **private** (its parent
groups are private, so it cannot be made public): every fetch, the script itself included, sends the
`DEPLOY-TOKEN` header with a read-only GitLab **deploy token** — scope `read_package_registry` only,
nothing else (no repository, no API, no write); revocable at any time in the project's *Settings →
Repository → Deploy tokens*. It lives in OpenBao at `monky/monky-tenancy/deployd-download` (key
`token`); the monky-tenancy install kit carries it and passes `--token`, the ansible role sends it
from the vaulted `monky_deployd_download_token`. The one-liner:
```sh
curl -sSf -H "DEPLOY-TOKEN: $T" https://scm.tikali.ai/api/v4/projects/69/packages/generic/monky-deployd/<ver>/install.sh \
| sudo bash -s -- --env <id> --site <site> --token "$T" --bootstrap-file bootstrap.jwt
```
`install.sh` never prints the token (it goes through a 0600 curl `-K` file that is deleted after the
download; xtrace is switched off). A 401 on the download means the token is missing, revoked or
lacks the scope. Inside the estate `gitea.cbs.tikali.net` is split-horizon to jump1's RED EIP
(`10.10.0.175`), which has no HTTP ingress, so backend boxes cannot fetch from the Gitea mirror
(cbs/iac#102). Off-estate, `install.sh --source gitea` (ansible: `monky_deployd_base_url`/`_deb_url`,
no token) uses the Gitea release instead. The tag pipeline publishes to both (`release`, `release:gitea`).
## Publishing a release to Gitea by hand
When `release:gitea` is manual (no `GITEA_TOKEN` in CI) — from jump1 with the Gitea API token:
@@ -100,10 +124,10 @@ for f in monky-deployd_0.1.0_amd64.deb monky-deployd_0.1.0_amd64.deb.sha256 inst
```
The assets are then at `https://gitea.cbs.tikali.net/mdella/monky-deployd/releases/download/v0.1.0/<file>`,
which is what `install.sh` fetches (GitLab artifacts from the tag pipeline's `package` job).
which is what `install.sh --source gitea` fetches (GitLab artifacts from the tag pipeline's `package` job).
## Ansible (osg1-07)
`ansible/roles/monky_deployd/` is the role skeleton to copy into osg1-07: `.deb` from the Gitea
release (sha256-verified), config template, bootstrap grant from a vault var, ACL on the identity,
`ansible/roles/monky_deployd/` is the role skeleton to copy into osg1-07: `.deb` from the
scm.tikali.ai package registry (sha256-verified), config template, bootstrap grant from a vault var, ACL on the identity,
timer, one tick. Rolled to env-dev-06..09 after the env-qa-02 pilot; env-dev-01 last.
+19 -12
View File
@@ -70,7 +70,8 @@ same tick, otherwise it exits 1 and says "re-run the install kit".
```json
{"env_id": "env-qa-02", "desired_sha": "7a10…", "action": "apply", "purge_volumes": false,
"bundle_url": "/v1/agent/bundle/env-qa-02/7a10…", "checkin_interval_s": 60,
"vault": {"addr": "https://bao.cbs.tikali.net:8200", "mount": "monky", "prefix": "env-qa-02/see"}}
"vault": {"addr": "https://bao.cbs.tikali.net:8200", "mount": "monky", "prefix": "env-qa-02/see",
"auth_mount": "jwt-tenancy", "auth_role": "see-env"}}
```
`action`: `apply` (desired ≠ applied), `none` (converged → heartbeat), `down` (retire; `purge_volumes`
is only meaningful here). `vault.mount` is the **KV** mount; the agent adopts it if it differs from
@@ -109,12 +110,16 @@ address, pull, renderer, images_policy, secrets_provider, agent{…}, files[]`.
### `POST /v1/agent/lease`
```json
{"env_id": "env-qa-02", "reason": "apply"} // reason: apply | renew
{"env_id": "env-qa-02"}
```
(The agent also sends `"reason": "apply" | "renew"` for its own logs; tenancy's `AgentLease` is
`{env_id}` and ignores unknown fields.)
```json
{"env_id": "env-qa-02", "login_jwt": "eyJ…", "ttl_s": 3600, "mount": "jwt-tenancy", "role": "see-env",
"vault": {"addr": "https://bao.cbs.tikali.net:8200", "mount": "monky"}}
"addr": "https://bao.cbs.tikali.net:8200"}
```
`addr` is the OpenBao address for the login below, top-level (tenancy `AgentLeaseOut`); the KV
mount/prefix come from `checkin`'s `vault`, not from the lease.
Then `POST /v1/auth/{mount}/login {"role": "{role}", "jwt": "{login_jwt}"}` on OpenBao. A body with
`wrapping_token` / `role_id` (the pre-Gate-1 AppRole lease) is refused with `LEASE_SHAPE` → report
`failed`. `429 LEASE_RATE_LIMITED` → exit 75.
@@ -133,15 +138,17 @@ first 2 KiB land in the audit log — it has been through the redactor.
## Divergences (2026-09-05)
- **monky-tenancy `main` (MR !15) still implements the AppRole lease and install kit**
(`AgentLeaseOut{wrapping_token, role_id}`, `bootstrap.wrap`, `bao.approle` in the kit's config). The
binding design is the plan's Gate 1 RESULT / ADR-0028 amendment: `{login_jwt, ttl_s, mount, role}`
and `POST /v1/auth/jwt-tenancy/login`. This agent implements the latter; against an un-migrated
tenancy it reports `failed` with `LEASE_SHAPE` and refuses `bao.approle` in its config. The tenancy
follow-up (deploy-grant signer, JWKS, `lease` shape, kit → `bootstrap.jwt`) is tracked on
monky-tenancy.
- The kit's generated config uses `tenancy.base_url: http://monky.tenancy.deploy:8081` — accepted as
an alias for `tenancy.{scheme,host,port}`.
- **Resolved 2026-09-05 (before v0.1.0 was tagged):** monky-tenancy !17 (`288df791`) landed the
tenancy side of the Gate 1 RESULT / ADR-0028 amendment — `AgentLeaseOut{env_id, login_jwt, ttl_s,
mount, role, addr}`, the ES256 deploy-grant signer (`app/agent_keys.py`), `GET
/.well-known/agent-jwks.json`, no AppRole and no response wrapping anywhere in tenancy. The
`LEASE_SHAPE` refusal of a `wrapping_token`/`role_id` body stays in this agent as a guard against a
stale tenancy, not as a description of `main`.
- **Resolved 2026-09-05:** the kit no longer generates a config file. Since monky-tenancy !22
(`61bd0281`) the one-time install script stages the bootstrap grant and runs `install.sh --env …
--site … --version … --bootstrap-file …`, and `install.sh` writes `/etc/monky-deployd/config.yaml`
(tenancy service name, OpenBao address and `jwt-tenancy`/`see-env`). `tenancy.base_url:` remains an
accepted alias for `tenancy.{scheme,host,port}` (`config.py`) for hand-written configs.
- `report` gains an optional `detail` (doc 24 §3.3); tenancy's `AgentReport` ignores unknown fields
today — if `strict` bodies land, `detail` folds into `log_tail`.
- Bundle sha header: tenancy sends `X-Bundle-Sha`, doc 24 says `X-Bundle-Sha256`; the agent reads
+1 -1
View File
@@ -4,4 +4,4 @@ Dials monky-tenancy over the mesh with the box's host identity, fetches the rend
leases a deploy grant, logs in to OpenBao, reads its own secrets, runs `docker compose`,
reports. Stdlib only; the optional `openziti` SDK is the `sdk` transport."""
__version__ = "0.1.0"
__version__ = "0.1.5"
+8 -2
View File
@@ -144,12 +144,18 @@ def parse(data: bytes, *, max_bytes: int = 4 * 1024 * 1024) -> Bundle:
# --- refusal checks (pure; names only, never values) -----------------------------------------
def _code_lines(text: str) -> str:
"""Drop comment lines: a `# … ${VAR} …` remark in .env.template (the renderer writes one)
is not a reference. Compose/dotenv comments start with `#` after optional whitespace."""
return "\n".join(ln for ln in text.splitlines() if not ln.lstrip().startswith("#"))
def referenced_vars(text: str) -> set[str]:
return {m.group(1) for m in _VAR_RE.finditer(text)}
return {m.group(1) for m in _VAR_RE.finditer(_code_lines(text))}
def defaulted_vars(text: str) -> set[str]:
return {m.group(1) for m in _VAR_DEFAULTED_RE.finditer(text)}
return {m.group(1) for m in _VAR_DEFAULTED_RE.finditer(_code_lines(text))}
def unresolved_vars(bundle: Bundle, provided: set[str]) -> list[str]:
+4 -2
View File
@@ -4,7 +4,7 @@ Bearer = the agent's OpenBao token (from the `jwt-tenancy` login). Tenancy pins
the token's `meta.env_id` (403 AGENT_ENV_MISMATCH -> exit 78, never retried) and refuses a token
whose deploy grant was superseded (401 AGENT_UNAUTHENTICATED -> re-bootstrap or re-run the kit).
Lease shape of record (Gate 1 v2, 2026-09-05): `{login_jwt, ttl_s, mount, role, vault}`. An
Lease shape of record (Gate 1 v2, 2026-09-05): `{env_id, login_jwt, ttl_s, mount, role, addr}`. An
AppRole-era body (`wrapping_token`, `role_id`) is refused loudly there is nothing to unwrap."""
from __future__ import annotations
@@ -158,13 +158,15 @@ class TenancyClient:
"(ADR-0028 amendment 2026-09-05)",
)
raise TenancyError(200, "LEASE_SHAPE", "lease response carries no login_jwt")
# shape of record (tenancy AgentLeaseOut): `addr` is top-level; a pre-0.1.x `vault{}` object
# is still read as a fallback so an older fake or tenancy does not break the lease
v = js.get("vault") or {}
return Lease(
login_jwt=str(js["login_jwt"]),
ttl_s=int(js.get("ttl_s") or 3600),
mount=str(js.get("mount") or "jwt-tenancy"),
role=str(js.get("role") or "see-env"),
vault=Vault(addr=v.get("addr"), mount=v.get("mount"), prefix=v.get("prefix")),
vault=Vault(addr=js.get("addr") or v.get("addr"), mount=v.get("mount"), prefix=v.get("prefix")),
)
def report(
+8
View File
@@ -104,6 +104,14 @@ class SdkTransport(Transport):
return socket.create_connection((host, port), timeout=timeout)
except OSError as exc:
raise TransportError(f"transport sdk: dial {host}:{port} failed: {exc}") from exc
except Exception as exc: # noqa: BLE001 - the SDK raises bare Exception((code, msg)) and TypeError
# openziti-sdk-py: an address with NO matching intercept falls through to
# PySocket.connect(tuple) → TypeError; a matching intercept the identity may not dial
# raises Exception((-18, 'service not available')) — env-qa-02 pilot, 2026-09-07.
raise TransportError(
f"transport sdk: dial {host}:{port} failed: {exc} — no intercept for that host:port, or this "
"identity has no dial policy for the service (check the intercept port and the identity's attrs)"
) from exc
def describe(self) -> str:
return f"sdk(identity={self.identity_path})"
+115 -32
View File
@@ -1,17 +1,26 @@
#!/usr/bin/env bash
# monky-deployd installer — Ubuntu 26.04 (verified target).
#
# curl -fsSL https://gitea.cbs.tikali.net/mdella/monky-deployd/raw/branch/main/packaging/install.sh \
# | sudo bash -s -- --env env-qa-02 --site cbs [--transport sdk|proxy|system] [--version 0.1.0] \
# [--enrol-jwt /path/monky-host.env-qa-02.jwt] [--laptop] < bootstrap.jwt
# T=<read-only GitLab deploy token, scope read_package_registry> # from the install kit / OpenBao
# curl -sSf -H "DEPLOY-TOKEN: $T" \
# https://scm.tikali.ai/api/v4/projects/69/packages/generic/monky-deployd/0.1.5/install.sh \
# | sudo bash -s -- --env env-qa-02 --site cbs --token "$T" --bootstrap-file bootstrap.jwt \
# [--transport sdk|proxy|system] [--version 0.1.5] [--enrol-jwt /path/monky-host.env-qa-02.jwt] \
# [--laptop] [--source gitlab|gitea] [--docker-data-root /home/docker-data]
#
# stdin (or --bootstrap-file): the ONE-TIME bootstrap deploy grant from the install kit
# (GET /v1/backends/{id}/agent/install). The enrolment JWT is read from --enrol-jwt or
# /etc/monky-deployd/enrol.jwt and is only needed when the host identity is not enrolled yet.
# --token / MONKY_DEPLOYD_TOKEN: the GitLab project is PRIVATE (its parent groups are private, so it
# cannot be made public); every download from the generic package registry — this script included —
# sends `DEPLOY-TOKEN: <token>`. The token is read-only (read_package_registry), revocable, and is
# never printed by this script. Not needed with --source gitea.
# --bootstrap-file (or stdin when the script runs from a FILE): the ONE-TIME bootstrap deploy grant
# from the install kit (GET /v1/backends/{id}/agent/install). The enrolment JWT is read from
# --enrol-jwt or /etc/monky-deployd/enrol.jwt and is only needed when the host identity is not
# enrolled yet.
#
# What it does (idempotent):
# 1. apt: ziti-edge-tunnel (OpenZiti `jammy` suite) if absent, docker-compose-plugin, acl
# 2. downloads the pinned monky-deployd_<ver>_amd64.deb + .sha256 from the Gitea release, verifies, installs
# 2. downloads the pinned monky-deployd_<ver>_amd64.deb + .sha256 from the scm.tikali.ai package registry
# (DEPLOY-TOKEN header; --source gitea: the Gitea release, off-estate), verifies, installs
# 3. enrols /opt/openziti/etc/identities/monky-host.<env>.json if absent (ziti-edge-tunnel enroll),
# chown ziti:ziti 0600, switches ziti-edge-tunnel.service to `run-host` via a drop-in
# 4. writes /etc/monky-deployd/config.yaml, ACLs so user monky-deployd can read the identity,
@@ -20,14 +29,25 @@
set -euo pipefail
umask 077
DEFAULT_VERSION="0.1.0"
BASE_URL="${MONKY_DEPLOYD_BASE_URL:-https://gitea.cbs.tikali.net/mdella/monky-deployd}"
DEFAULT_VERSION="0.1.5"
# Download source. PRIMARY is the GitLab project's generic package registry on scm.tikali.ai: inside
# the estate gitea.cbs.tikali.net is split-horizon to jump1's RED EIP (10.10.0.175), which has no HTTP
# ingress, so backend boxes cannot reach the Gitea mirror (cbs/iac#102); scm.tikali.ai they can. The
# project is PRIVATE (parent groups are private), so the registry needs the read-only deploy token
# (--token / MONKY_DEPLOYD_TOKEN, sent as the DEPLOY-TOKEN header). `--source gitea` keeps the Gitea
# release as the off-estate alternative (no token). --base-url / MONKY_DEPLOYD_BASE_URL override the
# base for whichever layout is selected.
GITLAB_BASE_URL="https://scm.tikali.ai/api/v4/projects/69/packages/generic/monky-deployd"
GITEA_BASE_URL="https://gitea.cbs.tikali.net/mdella/monky-deployd"
SOURCE="${MONKY_DEPLOYD_SOURCE:-gitlab}"
BASE_URL="${MONKY_DEPLOYD_BASE_URL:-}"
TOKEN="${MONKY_DEPLOYD_TOKEN:-}"
OPENZITI_SUITE="${OPENZITI_SUITE:-jammy}"
IDENTITY_DIR="/opt/openziti/etc/identities"
ETC="/etc/monky-deployd"
ENV_ID="" SITE="" TRANSPORT="sdk" VERSION="$DEFAULT_VERSION" ENROL_JWT="" BOOTSTRAP_FILE="" NO_RUN="" FORCE_CONFIG="" BAO_CA="" LAPTOP="false"
ENV_ID="" SITE="" TRANSPORT="sdk" DOCKER_DATA_ROOT="" TUNNEL_FAILED="" DEPLOYD_VERSION="$DEFAULT_VERSION" ENROL_JWT="" BOOTSTRAP_FILE="" NO_RUN="" FORCE_CONFIG="" BAO_CA="" LAPTOP="false"
usage() { sed -n '2,20p' "$0"; exit "${1:-0}"; }
usage() { if [ -f "$0" ]; then sed -n '2,28p' "$0"; else echo "monky-deployd install.sh — see README.md (Install)"; fi; exit "${1:-0}"; }
die() { echo "install.sh: $*" >&2; exit 1; }
log() { echo "==> $*"; }
@@ -36,11 +56,14 @@ while [ $# -gt 0 ]; do
--env) ENV_ID="$2"; shift 2 ;;
--site) SITE="$2"; shift 2 ;;
--transport) TRANSPORT="$2"; shift 2 ;;
--version) VERSION="$2"; shift 2 ;;
--version) DEPLOYD_VERSION="$2"; shift 2 ;;
--docker-data-root) DOCKER_DATA_ROOT="$2"; shift 2 ;;
--enrol-jwt) ENROL_JWT="$2"; shift 2 ;;
--bootstrap-file) BOOTSTRAP_FILE="$2"; shift 2 ;;
--bao-ca) BAO_CA="$2"; shift 2 ;;
--base-url) BASE_URL="$2"; shift 2 ;;
--source) SOURCE="$2"; shift 2 ;;
--token) TOKEN="$2"; shift 2 ;;
--laptop) LAPTOP="true"; shift ;;
--no-run) NO_RUN=1; shift ;;
--force-config) FORCE_CONFIG=1; shift ;;
@@ -56,13 +79,21 @@ done
SITE="${SITE,,}"
[[ "$SITE" =~ ^(cbs|pdx)$ ]] || die "site must be cbs|pdx"
[[ "$TRANSPORT" =~ ^(sdk|proxy|system)$ ]] || die "transport must be sdk|proxy|system"
[[ "$SOURCE" =~ ^(gitlab|gitea)$ ]] || die "source must be gitlab|gitea"
# never let the token leak through xtrace / the environment of children
{ set +x; } 2>/dev/null
export -n MONKY_DEPLOYD_TOKEN 2>/dev/null || true
IDENTITY="$IDENTITY_DIR/monky-host.$ENV_ID.json"
export DEBIAN_FRONTEND=noninteractive
if [ -r /etc/os-release ]; then
. /etc/os-release
if [ "${ID:-}" != "ubuntu" ] || [ "${VERSION_ID:-}" != "26.04" ]; then
echo "WARNING: verified on Ubuntu 26.04; this is ${PRETTY_NAME:-unknown}. Continuing." >&2
# read os-release in a SUBSHELL: it defines VERSION/NAME/ID/... and sourcing it inline clobbered
# this script's own variables (0.1.2 → "26.04 LTS (Resolute Raccoon)" on the env-qa-02 pilot)
OS_ID=$(. /etc/os-release; printf '%s' "${ID:-}")
OS_VERSION_ID=$(. /etc/os-release; printf '%s' "${VERSION_ID:-}")
OS_CODENAME=$(. /etc/os-release; printf '%s' "${VERSION_CODENAME:-${UBUNTU_CODENAME:-}}")
if [ "$OS_ID" != "ubuntu" ] || [ "$OS_VERSION_ID" != "26.04" ]; then
echo "WARNING: verified on Ubuntu 26.04 only (this is ${OS_ID:-?} ${OS_VERSION_ID:-?}); continuing" >&2
fi
fi
@@ -82,7 +113,34 @@ fi
if [ "$TRANSPORT" = "proxy" ] && ! command -v ziti >/dev/null 2>&1; then
need_pkgs+=(openziti) # the `ziti` CLI (ziti tunnel proxy) from the same repo
fi
command -v docker >/dev/null 2>&1 || die "docker is not installed; install Docker Engine first (https://docs.docker.com/engine/install/ubuntu/)"
if ! command -v docker >/dev/null 2>&1; then
# Docker Engine from Docker's apt repo (the suite for this Ubuntu codename); falls back to Ubuntu's
# docker.io if Docker has no suite for the codename yet. --docker-data-root puts the data-root on
# a dedicated disk BEFORE the daemon first starts (pilot VMs mount one at /home/docker-data).
log "docker is not installed — installing Docker Engine"
export DEBIAN_FRONTEND=noninteractive
install -d -m 0755 /etc/apt/keyrings
if curl -fsSL --max-time 20 "https://download.docker.com/linux/ubuntu/dists/${OS_CODENAME:-noble}/Release" -o /dev/null 2>/dev/null; then
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
chmod a+r /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu ${OS_CODENAME:-noble} stable" > /etc/apt/sources.list.d/docker.list
apt-get update -qq
docker_pkgs="docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin"
else
log "Docker has no apt suite for '${OS_CODENAME:-?}' yet — using Ubuntu's docker.io"
apt-get update -qq
docker_pkgs="docker.io docker-compose-v2"
fi
if [ -n "$DOCKER_DATA_ROOT" ]; then
install -d -m 0710 "$DOCKER_DATA_ROOT"
install -d -m 0755 /etc/docker
[ -f /etc/docker/daemon.json ] || printf '{ "data-root": "%s" }\n' "$DOCKER_DATA_ROOT" > /etc/docker/daemon.json
fi
# shellcheck disable=SC2086
apt-get install -y -qq --no-install-recommends $docker_pkgs
systemctl enable --now docker
log "installed $(docker --version) (data-root $(docker info -f '{{.DockerRootDir}}' 2>/dev/null || echo default))"
fi
docker compose version >/dev/null 2>&1 || need_pkgs+=(docker-compose-plugin)
python3 -c 'import sys; sys.exit(0 if sys.version_info >= (3, 12) else 1)' 2>/dev/null || die "python3 >= 3.12 is required"
if [ "${#need_pkgs[@]}" -gt 0 ]; then
@@ -93,15 +151,28 @@ fi
# --- 2. the pinned .deb -----------------------------------------------------------------------------
installed="$(dpkg-query -W -f='${Version}' monky-deployd 2>/dev/null || true)"
if [ "$installed" = "$VERSION" ]; then
log "monky-deployd $VERSION already installed"
if [ "$installed" = "$DEPLOYD_VERSION" ]; then
log "monky-deployd $DEPLOYD_VERSION already installed"
else
tmp="$(mktemp -d)"; trap 'rm -rf "$tmp"' EXIT
deb="monky-deployd_${VERSION}_amd64.deb"
url="$BASE_URL/releases/download/v${VERSION}"
log "downloading $deb from $url"
curl -fsSL -o "$tmp/$deb" "$url/$deb"
curl -fsSL -o "$tmp/$deb.sha256" "$url/$deb.sha256"
deb="monky-deployd_${DEPLOYD_VERSION}_amd64.deb"
# the deploy token travels in a 0600 curl config file (-K), never on the command line, so neither
# `ps` nor an xtrace shows it; the file dies with $tmp
curlrc="$tmp/curlrc"; : > "$curlrc"; chmod 0600 "$curlrc"
if [ "$SOURCE" = gitlab ]; then
url="${BASE_URL:-$GITLAB_BASE_URL}/${DEPLOYD_VERSION}"
if [ -n "$TOKEN" ]; then
printf 'header = "DEPLOY-TOKEN: %s"\n' "$TOKEN" > "$curlrc"
else
echo "WARNING: no --token/MONKY_DEPLOYD_TOKEN: the scm.tikali.ai project is private, the download will 401 (use --source gitea off-estate)" >&2
fi
else
url="${BASE_URL:-$GITEA_BASE_URL}/releases/download/v${DEPLOYD_VERSION}"
fi
log "downloading $deb from $url${TOKEN:+ (DEPLOY-TOKEN)}"
curl -fsSL -K "$curlrc" -o "$tmp/$deb" "$url/$deb" || die "download of $deb failed (private registry: is the deploy token set and valid?)"
curl -fsSL -K "$curlrc" -o "$tmp/$deb.sha256" "$url/$deb.sha256" || die "download of $deb.sha256 failed"
rm -f "$curlrc"
(cd "$tmp" && sha256sum -c "$deb.sha256") || die "sha256 mismatch on $deb"
apt_update_once || true
apt-get install -y -qq "$tmp/$deb"
@@ -123,14 +194,21 @@ chmod 0600 "$IDENTITY"
# ziti-edge-tunnel as a HOST (bind side, no tun/DNS) — the agent dials with the SDK or the proxy
install -d /etc/systemd/system/ziti-edge-tunnel.service.d
cat > /etc/systemd/system/ziti-edge-tunnel.service.d/run-host.conf <<'DROPIN'
# monky-deployd: run-host mode (no tun, no DNS); identities from the standard directory
# monky-deployd: run-host mode (no tun, no DNS); identities from the standard directory.
# ExecStartPre is cleared: the package's ziti-edge-tunnel.sh (auto-enrol *.jwt in the identity dir)
# is not executable by user ziti on 1.18.x (203/EXEC, env-qa-02 pilot) and run-host does not need it.
[Service]
ExecStartPre=
ExecStart=
ExecStart=/opt/openziti/bin/ziti-edge-tunnel run-host --identity-dir=/opt/openziti/etc/identities
DROPIN
systemctl daemon-reload
systemctl enable --now ziti-edge-tunnel.service
systemctl restart ziti-edge-tunnel.service || true
systemctl enable ziti-edge-tunnel.service >/dev/null 2>&1 || true
if ! systemctl restart ziti-edge-tunnel.service; then
# do NOT abort here: config, ACLs, grant and timer below must land so a re-run needs no new kit
echo "WARNING: ziti-edge-tunnel.service failed to start — see: journalctl -u ziti-edge-tunnel; continuing" >&2
TUNNEL_FAILED=1
fi
# --- 4. config, ACLs, bootstrap grant -----------------------------------------------------------------
install -d -m 0750 -o root -g monky-deployd "$ETC"
@@ -147,7 +225,7 @@ if [ -s "$ETC/config.yaml" ] && [ -z "$FORCE_CONFIG" ]; then
log "keeping existing $ETC/config.yaml (use --force-config to rewrite)"
else
cat > "$ETC/config.yaml" <<CFG
# written by install.sh $(date -u +%FT%TZ) — monky-deployd $VERSION
# written by install.sh $(date -u +%FT%TZ) — monky-deployd $DEPLOYD_VERSION
env_id: $ENV_ID
site: $SITE
transport: $TRANSPORT
@@ -155,7 +233,7 @@ identity: $IDENTITY
tenancy:
service: monky.tenancy.deploy
host: monky.tenancy.deploy
port: 8081
port: 443 # the service's INTERCEPT port (host.v1 forwards to 8081 inside the pod); plain HTTP inside the mesh
scheme: http
proxy_addr: 127.0.0.1:18443
bao:
@@ -179,10 +257,11 @@ if [ "$TRANSPORT" = "proxy" ]; then
printf 'ZITI_IDENTITY=%s\n' "$IDENTITY" > "$ETC/proxy.env"; chmod 0644 "$ETC/proxy.env"
systemctl enable --now monky-deployd-proxy.service
fi
# bootstrap grant: stdin (the kit pipes it) or --bootstrap-file; 0600, owned by the agent so it can consume it
# bootstrap grant: --bootstrap-file, or stdin — but ONLY when this script runs from a file: with
# `curl … | bash -s --` stdin IS the script, so reading it here would swallow the rest of it
grant=""
if [ -n "$BOOTSTRAP_FILE" ]; then grant="$(tr -d '[:space:]' < "$BOOTSTRAP_FILE")"
elif [ ! -t 0 ]; then grant="$(tr -d '[:space:]' </dev/stdin || true)"; fi
elif [ ! -t 0 ] && [ -f "$0" ]; then grant="$(tr -d '[:space:]' </dev/stdin || true)"; fi
if [ -n "$grant" ]; then
printf '%s\n' "$grant" > "$ETC/bootstrap.jwt"
chown monky-deployd:monky-deployd "$ETC/bootstrap.jwt"; chmod 0600 "$ETC/bootstrap.jwt"
@@ -192,7 +271,7 @@ elif [ -s /var/lib/monky-deployd/bao.token ]; then
else
echo "WARNING: no bootstrap grant on stdin and no bao.token: the agent cannot check in until you provide one" >&2
fi
unset grant
unset grant TOKEN
# --- 5. timer + first tick ----------------------------------------------------------------------------------
systemctl daemon-reload
@@ -206,7 +285,7 @@ rm -f "$ETC/enrol.jwt"
cat <<CHECK
monky-deployd $VERSION installed for $ENV_ID ($SITE, transport $TRANSPORT). Verification checklist (Ubuntu 26.04):
monky-deployd $DEPLOYD_VERSION installed for $ENV_ID ($SITE, transport $TRANSPORT). Verification checklist (Ubuntu 26.04):
python3 --version # >= 3.12
docker compose version # compose plugin present
systemctl is-active ziti-edge-tunnel # run-host mode (drop-in run-host.conf)
@@ -220,3 +299,7 @@ monky-deployd $VERSION installed for $ENV_ID ($SITE, transport $TRANSPORT). Veri
CHECK
CHECK_STATUS="$(monky-deployd status 2>&1 || true)"
echo "$CHECK_STATUS" | sed 's/^/ | /'
if [ -n "$TUNNEL_FAILED" ] && ! systemctl is-active --quiet ziti-edge-tunnel.service; then
echo "ERROR: ziti-edge-tunnel.service is not running; the agent cannot reach the mesh until it is. Everything else is installed — fix the unit and 'systemctl restart ziti-edge-tunnel monky-deployd'." >&2
exit 1
fi
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "monky-deployd"
version = "0.1.0"
version = "0.1.5"
description = "Monky backend pull agent: checkin -> bundle -> lease -> OpenBao -> docker compose -> report, over the ziti mesh (MONKY-ADR-0028)"
readme = "README.md"
requires-python = ">=3.12"
+2 -1
View File
@@ -354,7 +354,8 @@ class FakeTenancy:
"ttl_s": 3600,
"mount": self.bao.mount,
"role": self.bao.role,
"vault": {"addr": "https://bao.cbs.tikali.net:8200", "mount": self.kv_mount},
# tenancy AgentLeaseOut: `addr` is top-level; KV mount/prefix come from checkin
"addr": "https://bao.cbs.tikali.net:8200",
},
headers={"Cache-Control": "no-store"},
)
+2 -1
View File
@@ -7,6 +7,7 @@ import os
import time
from pathlib import Path
from monky_deployd import __version__
from monky_deployd import state as statemod
from monky_deployd.agent import EX_ENV_MISMATCH, EX_FAIL, EX_OK, EX_TEMPFAIL, Agent
from monky_deployd.bundle import bundle_sha
@@ -24,7 +25,7 @@ def test_first_tick_bootstraps_applies_and_reports(bootstrapped, tenancy, bao, f
assert not Path(cfg.bootstrap_path).exists()
assert cfg.token_path.exists() and (cfg.token_path.stat().st_mode & 0o777) == 0o600
# protocol: checkin -> bundle -> lease -> report applied
assert tenancy.checkins[0]["env_id"] == ENV and tenancy.checkins[0]["agent_version"] == "0.1.0"
assert tenancy.checkins[0]["env_id"] == ENV and tenancy.checkins[0]["agent_version"] == __version__
assert tenancy.checkins[0]["host"]["docker"] == "28.3.0"
assert len(tenancy.leases) == 1 and tenancy.leases[0]["reason"] == "apply"
assert [r["result"] for r in tenancy.reports] == ["applied"]
+13
View File
@@ -79,3 +79,16 @@ def test_disk_need_bytes_spellings():
assert b.parse(tar_bytes(make_files(meta={"agent": {"disk_need_bytes": 5}}))).disk_need_bytes == 5
assert b.parse(tar_bytes(make_files(meta={"disk": {"need_bytes": 7}}))).disk_need_bytes == 7
assert b.parse(tar_bytes(make_files())).disk_need_bytes == 0
def test_placeholders_in_comment_lines_are_not_references():
"""The renderer's .env.template header says '... substitutes every ${VAR} ...' — that must not
become an unresolved 'VAR' (env-qa-02 pilot: ENV_INCOMPLETE: unresolved: VAR)."""
from monky_deployd.bundle import defaulted_vars, referenced_vars
text = (
"# The on-box agent substitutes every ${VAR} from OpenBao per secrets.manifest.json.\n"
" # ${ALSO_COMMENT}\nGEMINI_API_KEY=${GEMINI_API_KEY}\nPG=${PGPASSWORD:-x}\n"
)
assert referenced_vars(text) == {"GEMINI_API_KEY", "PGPASSWORD"}
assert defaulted_vars(text) == {"PGPASSWORD"}
+2 -2
View File
@@ -1,7 +1,7 @@
import json
from pathlib import Path
from monky_deployd import cli
from monky_deployd import __version__, cli
from monky_deployd.agent import EX_OK
@@ -49,7 +49,7 @@ def test_bootstrap_command(bootstrapped, bao, capsys):
def test_version_and_bad_config(capsys, tmp_path):
assert cli.main(["version"]) == 0
assert capsys.readouterr().out.strip() == "0.1.0"
assert capsys.readouterr().out.strip() == __version__
bad = tmp_path / "c.yaml"
bad.write_text("env_id: nope\nsite: cbs\n")
assert cli.main(["-c", str(bad), "status"]) == 78