fix(packaging): an upgrade must not stop and disable the agent — 0.1.9

dpkg calls the OLD package's prerm on an UPGRADE as well as on a removal (rpm passes a
remaining-instance count), and preremove.sh ran `systemctl disable --now monky-deployd.timer`
unconditionally. Upgrading env-dev-01 and env-dev-08 to 0.1.8 today stopped and disabled both
agents.

The failure is silent, which is the dangerous part: the box stays reachable, the containers keep
running, and nothing reports that check-ins have ceased — the backend just stops converging. A
fleet upgrade would have taken every agent offline at once and looked like a success.

preremove.sh now returns early for every upgrade shape (upgrade, failed-upgrade, deconfigure,
rpm's 1) and only disables on a real removal. postinstall.sh try-restarts the long-lived proxy
unit so it picks up the new code; the timer needs nothing, since each tick is a fresh process.

Tests drive the script with a fake systemctl on PATH and assert an upgrade touches no units.
OPERATIONS.md warns that a box coming FROM 0.1.8 or earlier still needs its timer re-enabled by
hand, because the old prerm has already run by then.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KLB7jieMNRkTsJ2epr4Ds1
This commit is contained in:
2026-09-09 04:07:30 +00:00
parent 17a64843f1
commit 4ff2e29fbc
7 changed files with 77 additions and 3 deletions
+12
View File
@@ -1,6 +1,18 @@
<!-- xlate:verbatim-fences --> <!-- xlate:verbatim-fences -->
# Changelog # Changelog
## 0.1.9 — an upgrade no longer stops the agent (2026-09-09)
- **`dpkg -i` over a running agent disabled it.** dpkg calls the OLD package's `prerm` on an
**upgrade** as well as on a removal (rpm passes a remaining-instance count), and `preremove.sh`
ran `systemctl disable --now monky-deployd.timer` unconditionally. Upgrading env-dev-01 and
env-dev-08 from 0.1.6/0.1.7 to 0.1.8 stopped and **disabled** both agents. It is silent: the box
stays up, the containers keep running, and nothing reports that check-ins have ceased — the
backend simply stops converging. `preremove.sh` now returns early for every upgrade shape
(`upgrade`, `failed-upgrade`, `deconfigure`, rpm's `1`), and `postinstall.sh` `try-restart`s the
long-lived proxy unit so it picks up the new code. A fleet upgrade would have taken every agent
offline at once.
## 0.1.8 — onboarding: keep the identity readable, refuse a full disk (2026-09-09) ## 0.1.8 — onboarding: keep the identity readable, refuse a full disk (2026-09-09)
Three faults from one onboarding (env-dev-08, agent-managed, 2026-09-09), each of which sent the Three faults from one onboarding (env-dev-08, agent-managed, 2026-09-09), each of which sent the
+4
View File
@@ -30,6 +30,10 @@ the `ziti` **group**, plus an explicit ACL `u:monky-deployd:r` and a default ACL
> first refresh and then fails every tick (env-dev-08, 2026-09-09). On a host where the tunneller was > first refresh and then fails every tick (env-dev-08, 2026-09-09). On a host where the tunneller was
> installed independently of the kit, check `id monky-deployd` for `ziti` before anything else. > installed independently of the kit, check `id monky-deployd` for `ziti` before anything else.
> **Upgrading the agent.** `dpkg -i` keeps the timer enabled from 0.1.9 on. On a box upgraded from
> 0.1.8 or earlier the old package's `prerm` already ran and **disabled** it — check
> `systemctl is-enabled monky-deployd.timer` after the upgrade and `systemctl enable --now` it if needed.
## Reading the journal ## Reading the journal
| line | meaning | | line | meaning |
+1 -1
View File
@@ -4,4 +4,4 @@ Dials monky-tenancy over the mesh with the box's host identity, fetches the rend
leases a deploy grant, logs in to OpenBao, reads its own secrets, runs `docker compose`, leases a deploy grant, logs in to OpenBao, reads its own secrets, runs `docker compose`,
reports. Stdlib only; the optional `openziti` SDK is the `sdk` transport.""" reports. Stdlib only; the optional `openziti` SDK is the `sdk` transport."""
__version__ = "0.1.8" __version__ = "0.1.9"
+4 -1
View File
@@ -22,6 +22,9 @@ if getent group ziti >/dev/null; then usermod -a -G ziti monky-deployd || true;
/opt/monky-deployd/venv/bin/python -c 'import monky_deployd' || { echo "monky-deployd: venv unusable (python3 mismatch?)" >&2; exit 1; } /opt/monky-deployd/venv/bin/python -c 'import monky_deployd' || { echo "monky-deployd: venv unusable (python3 mismatch?)" >&2; exit 1; }
if [ -d /run/systemd/system ]; then if [ -d /run/systemd/system ]; then
systemctl daemon-reload || true systemctl daemon-reload || true
# do NOT enable the timer here: install.sh / the ansible role do it after the config exists # do NOT enable the timer here: install.sh / the ansible role do it after the config exists.
# An UPGRADE is different — the timer is already enabled and must keep running, so restart the
# long-lived proxy unit onto the new code. `try-restart` is a no-op when it is not running.
systemctl try-restart monky-deployd-proxy.service 2>/dev/null || true
fi fi
exit 0 exit 0
+9
View File
@@ -1,5 +1,14 @@
#!/bin/sh #!/bin/sh
set -e set -e
# dpkg calls the OLD package's prerm on an UPGRADE as well as on a removal, and rpm calls it with
# an install count. Disabling the timer unconditionally therefore stopped the agent on every
# upgrade and left it disabled — silently, because the box stays reachable and nothing else
# notices that check-ins have ceased (env-dev-01 and env-dev-08, 2026-09-09).
# dpkg: "$1" is `remove`, `upgrade <new-version>`, `deconfigure …` or `failed-upgrade`
# rpm : "$1" is the number of instances that will remain — 1 on upgrade, 0 on uninstall
case "${1:-}" in
upgrade | failed-upgrade | deconfigure | 1) exit 0 ;;
esac
if [ -d /run/systemd/system ]; then if [ -d /run/systemd/system ]; then
systemctl disable --now monky-deployd.timer 2>/dev/null || true systemctl disable --now monky-deployd.timer 2>/dev/null || true
systemctl disable --now monky-deployd-proxy.service 2>/dev/null || true systemctl disable --now monky-deployd-proxy.service 2>/dev/null || true
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "monky-deployd" name = "monky-deployd"
version = "0.1.8" version = "0.1.9"
description = "Monky backend pull agent: checkin -> bundle -> lease -> OpenBao -> docker compose -> report, over the ziti mesh (MONKY-ADR-0028)" description = "Monky backend pull agent: checkin -> bundle -> lease -> OpenBao -> docker compose -> report, over the ziti mesh (MONKY-ADR-0028)"
readme = "README.md" readme = "README.md"
requires-python = ">=3.12" requires-python = ">=3.12"
+46
View File
@@ -0,0 +1,46 @@
"""The maintainer scripts must not stop the agent on an upgrade.
dpkg calls the OLD package's prerm on an upgrade as well as on a removal, and rpm calls it with
an install count. `systemctl disable --now` there stopped the agent on every upgrade and left it
disabled — silently, because the box stays reachable and nothing else notices that check-ins have
ceased (env-dev-01 and env-dev-08, 2026-09-09).
"""
from __future__ import annotations
import shutil
import subprocess
from pathlib import Path
import pytest
PREREMOVE = Path(__file__).resolve().parents[1] / "packaging" / "scripts" / "preremove.sh"
def _run(arg: str, tmp_path: Path) -> list[str]:
"""Run preremove with a fake `systemctl` on PATH and report the calls it made."""
calls = tmp_path / "systemctl.log"
fake = tmp_path / "bin"
fake.mkdir(exist_ok=True)
(fake / "systemctl").write_text(f'#!/bin/sh\necho "$@" >> {calls}\nexit 0\n')
(fake / "systemctl").chmod(0o755)
env = {"PATH": f"{fake}:/usr/bin:/bin"}
r = subprocess.run(["sh", str(PREREMOVE), arg], env=env, capture_output=True, text=True)
assert r.returncode == 0, r.stderr
return calls.read_text().splitlines() if calls.exists() else []
@pytest.mark.skipif(not Path("/run/systemd/system").is_dir(), reason="needs a systemd host to reach the disable branch")
@pytest.mark.parametrize("arg", ["remove", "0"])
def test_a_real_removal_disables_the_units(arg, tmp_path):
assert any("disable" in c for c in _run(arg, tmp_path))
@pytest.mark.parametrize("arg", ["upgrade", "failed-upgrade", "deconfigure", "1"])
def test_an_upgrade_leaves_the_units_alone(arg, tmp_path):
assert _run(arg, tmp_path) == []
def test_the_script_is_shell_clean():
assert shutil.which("sh")
subprocess.run(["sh", "-n", str(PREREMOVE)], check=True)