fix(actual-budget): page when the finance backup loses its off-box copy #203

Merged

kblack0610 merged 1 commit from fix/finance-backup-offbox-pages into master

2026-08-26 15:44:33 +00:00

kblack0610 commented

2026-08-26 15:44:05 +00:00

Owner

Follow-up to #202, implementing the two decisions taken on it.

Page when the off-box copy fails

Every other backup job treats the NAS copy as best-effort and exits 0 without it. For the finance data that policy has no safety net: the local copy is a hostPath on the single node holding the RWO PVC, and there is no offsite tier, so a quietly failing off-box copy leaves the whole budget history one dead Pi away from gone.

actual-budget-backup now exits non-zero when the off-box copy fails, so KubeJobFailed pages. The local archive is already written by that point and is kept - verified on disk after a failing run, since the error message claims it.

This is a deliberate divergence from the other five jobs rather than drift, so it is commented at the call site and recorded in the runbook as a per-job failure-policy table.

backoffLimit: 2

Consequence of the above. Each attempt writes a fresh timestamped archive and re-runs the keep-30 prune, so the default backoffLimit of 6 would let a single night of NAS downtime push 7 same-day archives through the local set and squeeze out the older days the retention exists to protect. 3 attempts rides out a brief blip and still leaves 10 days of local depth.

Verification

Same stubbed-smbclient harness as #202, against real budget data:

Case	Before	Now
happy path	exit 0	exit 0, verified 559596 bytes
`cd` fails, file lands in share root	WARN, exit 0	exit 1
upload truncated	WARN, exit 0	exit 1
local archive after a failed run	-	present on disk

Offsite

Decided: LAN-only accepted for now, revisit later. Recorded in the runbook as an explicit dated decision with the exposure spelled out (a whole-house loss takes the finance history, the Vaultwarden export and the HA config together), so it reads as deferred rather than overlooked.

Follow-up to #202, implementing the two decisions taken on it. ## Page when the off-box copy fails Every other backup job treats the NAS copy as best-effort and exits 0 without it. For the finance data that policy has no safety net: the local copy is a hostPath on the single node holding the RWO PVC, and there is no offsite tier, so a quietly failing off-box copy leaves the whole budget history one dead Pi away from gone. `actual-budget-backup` now exits non-zero when the off-box copy fails, so `KubeJobFailed` pages. The local archive is already written by that point and is kept - verified on disk after a failing run, since the error message claims it. This is a deliberate divergence from the other five jobs rather than drift, so it is commented at the call site and recorded in the runbook as a per-job failure-policy table. ## backoffLimit: 2 Consequence of the above. Each attempt writes a fresh timestamped archive and re-runs the keep-30 prune, so the default backoffLimit of 6 would let a single night of NAS downtime push 7 same-day archives through the local set and squeeze out the older days the retention exists to protect. 3 attempts rides out a brief blip and still leaves 10 days of local depth. ## Verification Same stubbed-smbclient harness as #202, against real budget data: | Case | Before | Now | |---|---|---| | happy path | exit 0 | exit 0, verified 559596 bytes | | `cd` fails, file lands in share root | WARN, exit 0 | **exit 1** | | upload truncated | WARN, exit 0 | **exit 1** | | local archive after a failed run | - | present on disk | ## Offsite Decided: LAN-only accepted for now, revisit later. Recorded in the runbook as an explicit dated decision with the exposure spelled out (a whole-house loss takes the finance history, the Vaultwarden export and the HA config together), so it reads as deferred rather than overlooked.

kblack0610 added 1 commit

2026-08-26 15:44:05 +00:00

fix(actual-budget): page when the finance backup loses its off-box copy

checks / fleet-roster (pull_request) Successful in 7s

Details

checks / kustomize-builds (pull_request) Successful in 7s

Details

0a90ad534b

Deliberate exception to the house best-effort rule, decided 2026-08-25.

Every other backup job treats the NAS copy as best-effort and exits 0
without it. For the finance data that policy has no safety net: the local
copy is a hostPath on the single node holding the RWO PVC, and there is no
offsite tier, so a quietly failing off-box copy leaves the whole budget
history one dead Pi away from gone. The job now exits non-zero when the
off-box copy fails, so KubeJobFailed pages. The local archive is already
written by that point and is kept.

Also sets backoffLimit: 2. Each attempt writes a fresh timestamped archive
and re-runs the keep-30 prune, so the default of 6 would let one night of
NAS downtime push 7 same-day archives through the local set and squeeze out
the older days it exists to protect. 3 attempts rides out a blip and still
leaves 10 days of local depth.

Verified with the stubbed-smbclient harness: happy path still exits 0; a
failed `cd` and a truncated upload now exit 1 where they previously warned;
the local archive is present on disk after a failed run.

docs: records the LAN-only decision as deliberate rather than an oversight,
adds a per-job failure-policy table, and corrects the troubleshooting entry
that said an unreachable NAS is expected for every job.