ADR: Tailnet-per-environment (Prod / Test / Dev) with declarative node sharing

Status

Rejected. The homelab will not split Prod/Test/Dev into separate tailnets. Environment separation stays as it is today: a single tailnet with tag-based grants.

The investigation is retained in full because the rejection is the useful part — it documents why an appealing structural-isolation story doesn’t survive contact with an already-working Services deployment.

Superseded in direction by ADR-0007: Workload isolation in a single tailnet — access pools, which keeps the workload/homelab separation inside the tailnet as ACL-defined access pools, with a separate tailnet retained only as a deferred off-ramp. Its core finding — that a tailnet boundary pays off where paths should be closed anyway (non-homelab workload), not between homelab environments — still stands.

Context

Today the homelab runs a single tailnet with tag-based environment separation. Services are already the primary exposure mechanism for containers across Prod/Test/Dev — advertised via Docktail labels and ScaleTail sidecars, with per-service MagicDNS names, per tailscale.md. That part of the design is settled and this ADR does not revisit it.

What is not settled is the environment boundary. Per tailscale-grants.md, isolation between Prod/Test/Dev is enforced entirely by grant rules:

{"src": ["tag:prod"], "dst": ["tag:prod"], "ip": ["*:*"]}
{"src": ["tag:test"], "dst": ["tag:test"], "ip": ["*:*"]}
{"src": ["tag:dev"], "dst": ["tag:dev"], "ip": ["*:*"]}

That works, but the isolation is one policy-file edit deep. A bad merge, a mis-tagged node, or a fat-fingered tag:* destination collapses Prod/Test/Dev into one flat network. The blast radius of a policy mistake is the entire homelab.

Two Tailscale capabilities change the calculus:

1. Additional tailnets via the API. POST /api/v2/organizations/-/tailnets provisions a fully isolated tailnet under the same account, no console clicks. The response carries a tailnet-scoped OAuth client — creation is org-scoped, but administration and deletion require that per-tailnet credential (worked example).
2. Declarative node sharing (docs, alpha, waitlist-gated). Cross-tailnet access becomes a policy-file construct — externalTailnets plus grants referencing group://<alias>/<name> or tag://<alias>/<name>. It is explicitly designed for machine-to-machine access between trusted tailnets, which is exactly the homelab’s shape.

The question this ADR frames: is a tailnet boundary the right isolation primitive for the homelab’s environments, and is the cross-tailnet seam cheap enough to live with?

What a tailnet split does to the existing Services layer

Because Services are already the exposure model, the split’s real cost lands here rather than on greenfield design:

• Service names become tailnet-qualified. Every service currently resolves under one tailnet’s MagicDNS suffix. Three tailnets means three suffixes, and the naming convention documented in tailscale.md needs an environment-aware revision. OPEN: does plex.<prod-tailnet>.ts.net replace the current flat convention, or does an alias layer hide the split?
• httpsEnabled is per-tailnet and off by default. Any new tailnet needs the setting turned on before TLS-terminating services work at all — a provisioning step that doesn’t exist today because the current tailnet was configured once, long ago.
• Docktail/ScaleTail config becomes tailnet-scoped. Sidecars and label-driven advertisement authenticate to a specific tailnet. Auth keys, tags, and tagOwners all become per-tailnet concerns.
• Cross-environment service consumption is the seam. Anything in one environment consuming a service in another (observability, backups, promotion tooling) stops being an in-tailnet lookup and becomes an externalTailnets grant.

OPEN and worth answering early: whether a Service can be published across a tailnet boundary. If yes, the boundary contract is service-level and clean. If not, cross-environment access is node-and-port level, which is coarser than the exposure model already in use — a genuine regression against current practice, and arguably the strongest argument for staying on one tailnet.

Correction to an earlier framing

An earlier draft of this ADR analyzed the seam using link-based sharing (sharing docs) and concluded the model was largely unworkable — tags stripped, shares user-scoped so tagged machines can’t accept them, quarantine by default. Those constraints are real but belong to link-based sharing, not declarative node sharing. Declarative sharing operates at the policy level and supports tag references and machine-to-machine access directly. The two mechanisms coexist; picking the wrong one produces the wrong conclusion, which is why they’re separated explicitly below.

Drivers

The two sharing mechanisms

Getting this distinction right is most of the decision.

Consequence: declarative sharing is the mechanism this design should be built on. Link-based sharing remains the right tool only for the partner/guest case — handing one machine to one outside human.

Shape of the config

Double opt-in, per the declarative node sharing docs. Receiving tailnet declares what may be referenced:

{
  "externalTailnets": {
    "prod": {
      "externalID": "<prod-tailnet-id>",
      "allowIncomingConnections": false,
      "allowExternalReferencesTo": ["tag:komodo-controller"]
    }
  }
}

Sharing tailnet accepts the relationship and writes the grant:

{
  "externalTailnets": {
    "dev": { "externalID": "<dev-tailnet-id>", "allowIncomingConnections": true }
  },
  "grants": [
    { "src": ["tag://dev/komodo-controller"], "dst": ["tag:host"], "ip": ["8120"] }
  ]
}

A grant referencing something absent from the other side’s allowExternalReferencesTo is silently ignored — no error, just no connectivity. That failure mode needs a verification step in any runbook.

Options

Retained as the investigation record. Outcome noted on each.

Option A — Status quo: one tailnet, tag-based separation — CHOSEN

• Pros: zero migration; flat MagicDNS; one policy file; single console; no alpha dependency; Services and access model untouched.
• Cons: isolation is advisory, not structural; one bad grant flattens all environments; no per-environment audit boundary.

Option B — Three tailnets (Prod / Test / Dev), declarative sharing between them — rejected

• Pros: isolation is structural — a Dev policy mistake cannot reach Prod; per-tailnet audit logs, auth keys, and tagOwners; Dev becomes genuinely disposable; whole lifecycle is API-drivable (create, httpsEnabled, policy push, scoped auth keys); tags and m2m survive the boundary; double opt-in means neither side can unilaterally widen access.
• Cons: depends on an alpha, waitlist-gated feature; three policy files and three scoped OAuth credentials to manage; FQDNs become tailnet-qualified; silent-ignore failure mode on mismatched references; per-tailnet settings (notably HTTPS certs, off by default) must be provisioned explicitly; rewrites a working Services exposure surface.

Option C — Two tailnets: Prod isolated, Test+Dev together — rejected

• Pros: captures the blast-radius win that matters at two-thirds the overhead; Test↔Dev promotion stays in-tailnet.
• Cons: Test loses a hard boundary from Dev, so “does Test mirror Prod?” gets fuzzier; still carries the full alpha dependency for one seam — paying the same risk for less of the benefit.

Option D — One tailnet, hardened — CARRIED FORWARD (with A)

Policy-file CI with ACL tests in GitHub Actions, mandatory review on the policy file, ban tag:* destinations, and tighten per-service grants on the Services already in place.

• Pros: near-zero migration; no alpha dependency; converts soft isolation into tested isolation.
• Cons: still one flat trust domain; a mistake that passes tests still lands everywhere; audit boundary unchanged.

Decision

Rejected — keep the current single-tailnet structure with tag-based environment separation.

The structural-isolation argument is real, and declarative node sharing does make tailnet-per-environment technically achievable in a way link-based sharing never did. It is still the wrong trade for this homelab. Four reasons, in order of weight:

1. It breaks access that currently works, to solve a problem that hasn’t bitten

Services are already the exposure mechanism for most containers, and access works today — admin tiers, partner access, per-service MagicDNS names. A tailnet split rewrites that working surface: tailnet-qualified DNS suffixes, tailnet-scoped Docktail/ScaleTail sidecar auth, per-tailnet httpsEnabled, and per-tailnet auth keys and tagOwners. The failure it prevents — a fat-fingered grant flattening environments — has not actually occurred. Trading working access for hypothetical blast-radius reduction is the wrong direction.

2. Cross-boundary sharing is coarser than what’s already in place

The unresolved question from the investigation is decisive: if a Service cannot be published across a tailnet boundary, then cross-environment access degrades to node-and-port grants. That is coarser than the per-service exposure model already running. And if the shareable app has to physically live in the tailnet it’s shared from, containers move off the Prod Docker host — which drags in Komodo stack placement and the promotion path from ADR-0001. Either answer is worse than the status quo for a boundary nobody asked for.

3. The complexity is not proportionate to a single-operator homelab

Three tailnets means three policy files, three consoles, three sets of tailnet-scoped OAuth credentials, three auth-key rotations, and per-tailnet provisioning steps that don’t exist today. Add the silent-ignore failure mode — a grant referencing something absent from the other side’s allowExternalReferencesTo produces no error, just no connectivity — and routine debugging gets materially harder. Isolation that is expensive to operate gets worked around, and a worked-around boundary is worse than an honest tag.

4. The alpha dependency lands on the wrong path

Declarative node sharing is alpha and waitlist-gated. Under this ADR it would sit on the path between the Komodo controller and Prod periphery agents — meaning homelab manageability depends on an alpha feature. That inverts the risk this ADR set out to reduce.

What is kept

• Environment separation stays tag-based (tag:prod / tag:test / tag:dev) in one tailnet. No grants rewrite, no tag taxonomy change, no Services churn.
• Partner access stays link-based sharing, per the adversarial model in tailscale.md. Outside humans are not modeled as external tailnets.
• Policy-file CI with ACL tests is still worth doing (the former Option D) and is the only recommendation carried forward from this ADR. It converts advisory isolation into tested isolation at near-zero cost, and it is independently useful.

Future option: collapse three environments to two

Noted, not decided: the Prod/Test/Dev split could later collapse to Test and Prod only. Under ADR-0007, Dev’s remaining purpose is being reconsidered — if non-homelab work moves to a workload environment, a third homelab environment may not be earning its keep. That is a separate decision and deliberately out of scope here.

Where the tailnet boundary does belong

The investigation’s real finding is that the boundary axis was wrong. Splitting environments imposes a boundary across paths that need to stay open (admin, Komodo, observability, backups). Splitting workload from homelab puts the boundary where the paths should be closed anyway. See ADR-0007.

Cross-environment flows that made the split expensive

Each of these works today only because everything is one tailnet. Each would have needed an explicit externalTailnets/grant design — collectively, this list is the concrete form of reason #1 above:

• Komodo controller (NUC, tag:admin) → Prod/Test/Dev periphery agents — the canonical m2m case; technically workable via tag:// references, but it puts homelab manageability behind an alpha feature
• Promotion path DevDocker → Test → Prod (per ADR-0001, promotion is a file move; the network path is currently implicit and would have to become explicit)
• Observability scrape/push paths across environments — direction matters against allowIncomingConnections
• Backup flows to tag:storage NAS (UnRaid = Prod, Synology = Test)
• Admin access from the operator’s devices to all three — one identity in three tailnets, or a device per tailnet, neither cheap
• Exit node — one per tailnet, or one shared
• Partner/guest access — stays on link-based sharing regardless
• ProxMox host, which hosts the Dev VM but is itself infrastructure

Trade-offs that would have been accepted (and were judged too costly)

• Alpha-feature dependency on the critical path for cross-environment automation.
• Longer FQDNs and a documentation sweep to match.
• Per-tailnet provisioning steps that don’t exist today: httpsEnabled PATCH, tagOwners, auth keys, policy push.
• Tailnet-scoped OAuth credentials must be captured at creation — the org token cannot administer or delete a tailnet it created.
• Cross-environment automation gets deliberately harder. Intended for environments; unhelpful for admin paths.

Consequences / follow-on work

Because the split is rejected, most of the original follow-on work is moot. What remains:

• Add policy-file CI with ACL tests. The one carried-forward recommendation. Ban tag:* destinations, require review on the policy file, run tests in GitHub Actions.
• No changes to tailscale-grants.md or tailscale.md from this ADR — the tag taxonomy and Services naming convention stand as documented.
• Record the rejection so the option isn’t re-litigated from scratch; the mechanism comparison above is the reusable part.
• Continue in ADR-0007 for the workload boundary, where the provisioning artifact and waitlist question actually apply.

Open questions — resolved or relocated

The rejection closes most of these. Recorded so the option isn’t re-litigated from scratch:

1. Plan/cost implications of additional tailnets — moot here; relocated to ADR-0007.
2. Waitlist timing for declarative node sharing — no longer gating this decision; still worth joining for ADR-0007.
3. Quarantine semantics vs. allowIncomingConnections for declaratively shared nodes — unresolved; relocated to ADR-0007, where it actually matters.
4. Operator identity/devices across three tailnets — moot; admin access stays in one tailnet, which is the point of the rejection.
5. Komodo topology across a boundary — moot; controller and periphery agents stay in one tailnet.
6. Can a Service be published across a tailnet boundary? — unresolved and now load-bearing for ADR-0007. Answering it decides whether shareable apps can stay on their current host or must physically move.
7. Does Test lose meaning as a Prod rehearsal under Option C? — moot; Option C rejected. Related but distinct question of collapsing to Test+Prod is noted as a future option above.
8. Migration order — moot; no migration.
9. Synced groups (SCIM/Google) can’t be referenced across tailnets — relocated to ADR-0007; only relevant where a boundary exists.

References

Primary sources

• Declarative node sharing — the mechanism this ADR is built on. Alpha, waitlist-gated. externalTailnets, double opt-in, tag:///group:// external reference syntax, allowExternalReferencesTo, allowIncomingConnections. Also the source for the SCIM/Google synced-group limitation.
• summer-with-tailscale — worked, runnable examples for the API-driven path. Modules used here:
  • 01-tailnet-sandboxes — OAuth token exchange, POST /organizations/-/tailnets, and the tailnet-scoped OAuth credential that the org token cannot substitute for on delete
  • 02-tailnet-membership — tsnet embedded membership; relevant if services join tailnets directly rather than via host daemons
  • 03-declarative-sharing — externalTailnets config walkthrough and scoping external access by group/tag instead of blanket dst: ["*"]
  • 04-api-is-the-way — policy file as data via GET/POST /tailnet/-/acl, the httpsEnabled PATCH, scoped/tagged/short-lived auth keys, plus end-to-end.sh

Supporting

• Share your machines with other users — link-based sharing. Source of the tag-stripping, quarantine, and user-scoped constraints; retained here for the partner/guest case and to keep the two mechanisms distinguishable.
• Tailscale Services GA — cited for the GA-era additions relevant to a split (declarative on-node JSON config, per-service audit logs, ACL tests), not as an introduction to Services, which the homelab already runs
• Grants syntax
• Tailnet policy file
• Tailscale API reference

ADR: Workload isolation in a single tailnet — access pools

Status

Proposed. Direction settled: workload/dev traffic stays in the current tailnet, separated by ACLs organized as access pools. The separate-tailnet option is deferred to a named trigger, not discarded.

Extends ADR-0006, which rejected splitting Prod/Test/Dev into separate tailnets. That investigation’s useful finding was that the boundary axis was wrong, not the boundary idea. This ADR takes the next step of that finding: separation of workload from homelab is real, but the mechanism is pools within one boundary, with structural separation preserved as an off-ramp.

Context

Two distinct things are currently conflated inside one tailnet:

1. The homelab — Prod and Test infrastructure, private applications, the Komodo GitOps pipeline, storage, admin interfaces. Everything here is either operator-only or shared with the partner as a trusted user.
2. Dev — nominally a homelab environment, but in practice where non-homelab work happens. It has no promotion relationship worth preserving with Test/Prod for that class of work.

The earlier draft of this ADR weighed two mechanisms: a separate tailnet for workload (structural isolation, independent policy, clean audit boundary) against staying in the shared tailnet with tag separation (zero migration, isolation only advisory). The pivotal consideration — that a boundary’s alpha dependency sits on a path that is allowed to degrade, unlike ADR-0006’s admin/Komodo paths — is retained.

The judgment has since shifted on the trigger. The conditions that would justify a boundary — outside resources managing elements, sustained workload or external-party growth — are not real today. Building the structural split now pays complexity for a hypothetical, and it puts reachability that does not yet need to exist on an alpha, waitlist-gated feature (declarative node sharing). The boundary is kept as the off-ramp, priced and designed, but not built.

Decision

Single tailnet, workload separated by access pools. The structural split is deferred, not discarded.

1. One tailnet for now

Workload/Dev elements stay in the current tailnet, protected by ACLs rather than a boundary. Nothing new to provision, no per-tailnet settings (httpsEnabled, tagOwners, auth keys), and the Services/Docktail/ScaleTail exposure layer works exactly as today. External sharing, when it happens, stays link-based — one machine handed to one invited user — which covers ad-hoc guest access without committing to a separate tailnet.

2. Scope framing shifts from “homelab first” to “environment with pools”

The tailnet is no longer scoped as the homelab with everything else grafted on. It is the environment housing purpose-defined pools — provided services as one pool, development/workloads as another. The pool set is open: additional pools are named when a need clears the bar; none are pre-declared here (see Open Questions). This reframing drives everything that follows: nodes are organized by purpose AND access, and the ACL question becomes “which pools can which users and services reach.”

3. ACLs are pool-based, not environment-tag-based

The access model follows the pools: grants are written in terms of who may reach which pool, in which direction. Two concept principles, without enumerating the pools themselves:

• One pool per node. A node belongs to exactly one pool. Cross-pool reach is an explicit, named grant — never a second membership. This keeps the ACL table non-ambiguous: a node’s pool is its identity, and exceptions are visible rules rather than overlapping tags.
• Directional by default. Workload → provided services is the natural direction and the common grant (dev needs to reach the things the environment provides). Services → workload is a named exception, not part of the default posture. Reverse-direction access is the thing that erodes the separation, so it must always be a visible rule.

The pool taxonomy itself — names, membership, concrete grants — is deliberately unspecified in this ADR. It is a follow-on design task (see Consequences).

4. Separate tailnet = deferred off-ramp

If the environment grows and outside resources need to manage elements, the workload moves to a new set of devices in a separate tailnet — the homelab tailnet untouched. The earlier draft’s separate-tailnet analysis is retained as the off-ramp design: structural isolation, independent policy file, clean audit boundary, external-group sharing without exposing private applications. Its alpha dependency (declarative node sharing) is acceptable once the boundary is real, because it then sits on a workload path that is allowed to degrade — the same risk posture, just correctly timed.

The pool taxonomy chosen now is the seam that cut will follow: pools map cleanly to future tailnet membership, and a split becomes a placement/policy exercise rather than a redesign. Policy-file CI should be in place first — a split that is mostly policy movement is only cheap if the policy is already tested.

Why this succeeds where ADR-0006 failed

Future option: separate tailnet (deferred, priced)

Noted, not built. When the trigger fires — outside resources managing elements, or sustained external-party growth — execute the off-ramp: new device set in a separate tailnet, provisioned via the API chain (OAuth exchange → create tailnet → httpsEnabled → tagOwners/policy → scoped auth keys), homelab tailnet untouched, cross-boundary access via declarative node sharing. Joining the waitlist for declarative sharing is only needed when the trigger becomes real; doing it now starts a clock for a feature nothing active depends on.

Open questions

1. Pool taxonomy — deliberately unspecified here. The concrete pools, names, membership, and grants are the follow-on design task. Implicit constraint from this ADR: no “miscellaneous” pool; a catch-all pool is how pools rot into a flat network.
2. Does the partner need any workload access? Default assumption: no — workload pools are operator-only. Link-based sharing remains available per-need; the answer affects whether the future split needs external-group modeling at all.
3. Exit node — does workload need its own, or use none?
4. Split trigger threshold — what concretely constitutes “outside resources managing elements”? Naming the threshold now makes the off-ramp decision unemotional later.
5. Quarantine semantics / synced groups — relocated to the off-ramp; only apply once a boundary exists.

Consequences / follow-on work

• Design the pool taxonomy and draft the HuJSON policy — tracked TODO, not done here. A policy skeleton with the pool structure and directional grants, reviewed against the principles above.
• Policy-file CI with ACL tests — now more load-bearing than under either prior framing: pools keep isolation advisory, and CI is what converts it into tested isolation. Ban tag:* destinations, require review on the policy file, map pool grants to tests, run in GitHub Actions.
• Update tailscale-grants.md once the pool taxonomy is concrete; convert the environment-tag framing to pool framing.
• Update tailscale.md to document workload pools, link-based sharing for guests, and the deferred-split framing.
• Do not join the declarative-sharing waitlist yet — retracted from the earlier draft; only needed once the off-ramp trigger is real.
• Rename Dev → Workload in homelab docs, and be explicit that it is outside the promotion chain, so it isn’t mistaken for a pipeline stage.
• Revisit ADR-0001: if Dev leaves the promotion chain, the per-environment ResourceSync branch layout has one fewer environment to steer.

References

• ADR-0006: Tailnet-per-environment — the rejected environment split; contains the link-based vs. declarative sharing mechanism comparison and the API provisioning details
• Declarative node sharing — alpha, waitlist-gated; the off-ramp’s cross-boundary mechanism, not active today
• Share your machines with other users — link-based sharing; the active mechanism for external access under this ADR
• summer-with-tailscale — worked examples for the API-driven tailnet provisioning chain the off-ramp will use
• tailscale.md — current Services, Docktail/ScaleTail, and access-tier design
• tailscale-grants.md — current tag taxonomy and grant rules

ADR: Orchestration Model — Docker-native vs Kubernetes/ArgoCD for the Homelab

Context

The homelab currently standardizes on Docker (rootful, on Fedora Server baremetal hosts) managed through Komodo for GitOps. The prior narrative decision (“Komodo over Kubernetes (K3s + ArgoCD)” in planning/decisions.md) chose Docker-native Komodo over a K3s + ArgoCD stack on the grounds of complexity, risk tolerance, and concept alignment with existing familiarity with compose files.

This ADR reopens that axis. Two things have shifted the picture:

1. Declarative lifecycle is now a hard requirement, not a nice-to-have. The goal is to easily and declaratively spin up / tear down / manage resources across environments. That is exactly what native GitOps (ArgoCD reconciling manifests) does out of the box — and what Komodo does with more manual steering (ResourceSync branch gymnastics, per-environment TOML edits on merge, see ADR 0001).
2. ArgoCD is the more common / more documented route than Komodo. If the operational surface is “declare desired state, let a controller reconcile,” ArgoCD is the industry-default implementation. Komodo gets us most of the way but is a smaller ecosystem with rougher edges (Podman compat, stack-status quirks).

So the earlier “Kubernetes is overkill for a homelab” conclusion is being stress-tested against a simpler question: is the thing we’re optimizing for simplicity of the runtime or simplicity of declarative management? Those pull in different directions.

The tension

Options considered

Option A: All Docker-native (Komodo)

Standardize everything on Docker + Komodo ResourceSync.

• Pros: lowest runtime complexity; builds on existing Compose investment; Dev OCP cluster still provides separate K8s skills.
• Cons: declarative management is real but requires the merge-hygiene rituals from ADR 0001; smaller ecosystem; no first-class “one command spins up the whole graph.”

Option B: All Kubernetes (K3s/K8s + ArgoCD)

Replace Komodo with a K3s or K8s control plane and ArgoCD as the GitOps controller for every workload.

• Pros: single, ubiquitous declarative model; ArgoCD is the common, well-documented path; spin up/down is a Git commit; one pane of glass for everything.
• Cons: real operational complexity (control plane, CRDs, more moving parts); the original “overkill for homelab” risk is still valid; more failure modes to own 24/7.

Option C: Split model (leading direction, not yet decided)

Use Docker for “prod apps” and Kubernetes/ArgoCD for “workload apps.”

• Keeps the low-friction Docker path for stable, user-facing services (Plex, etc.) where runtime simplicity matters most.
• Puts the heavier / more dynamic / more experiment-driven workloads on K8s/ArgoCD where declarative reconciliation pays off and aligns with work skills.
• The only clear advantage of “all Kubernetes” (Option B) over this split is the single pane of glass — ArgoCD as the one view of everything. That is a real UX win, but it is currently the only decisive argument for unifying onto K8s, and it may not outweigh the standing complexity cost.

Decision

No conclusion yet, this ADR is intentionally left open.

The leading direction is Option C (split model), driven by the desire for easy declarative management without forcing the entire homelab onto a K8s control plane it may not need. The single strongest argument for abandoning the split and going all-in on Kubernetes (Option B) is the unified ArgoCD dashboard; that has not yet been judged worth the operational cost.

What would move this from Draft to a decision:

• A concrete definition of which apps count as “prod apps” (Docker) vs “workload apps” (K8s); the boundary is currently intuitive, not enumerated.
• A validated K3s + ArgoCD bring-up path on the homelab hardware (the original decision flagged K3s-in-prod as risky; that risk needs a fresh look).
• A judgment on whether “single pane of glass” is worth the control-plane burden, or whether two dashboards (Komodo + ArgoCD) is acceptable.

Consequences (potential)

If Option C is adopted:

• Two management planes to operate (Komodo for Docker, ArgoCD for K8s) — more tooling surface, but each stays within its comfort zone.
• Prod apps keep the low-ceremony Docker/Compose lifecycle.
• Workload apps get first-class declarative reconciliation and stronger skills transfer to the work OCP stack.
• Loses the “one ArgoCD view of everything” that Option B would provide.

If Option B is adopted instead:

• Single declarative model and dashboard for the whole homelab.
• Pays the standing K3s operational cost the original decision sought to avoid.
• Supersedes the “Komodo over Kubernetes” narrative in decisions.md — that section would need revisiting.

Open Questions

• Where exactly is the “prod app” vs “workload app” line drawn?
• Does “prod app” vs “workload app” need to be an treated as alternative approach to my originally declared “Prod/Test/Dev” environments from ADR 0001?
  • Idea: If this is the case, it would call into question the order of evolution in regards to hardware, with “Prod” still running on a dedicated device, having “Test” running within the ProxMox cluster, and having “Workload/Dev” on a larger version of the OCP cluster.
• Is the K3s-in-prod risk from the original decision still valid, or has the operational picture changed?
• If my personal work is focused largely on kubernetes, will it be better to have everything involved with kubernetes, or will having “prod apps” in docker (option C) encourage me to not mess with prod apps and focus my time on just the kubernetes side of the field?
• Is there a “single pane of glass” option that can manage both docker and kubernetes, or at least have visibility on both?
• Does losing a single pane of glass (under the split) actually hurt day-to-day, or is it tolerable?
• Does this reopen the runtime question (ADR 0003 / 0002) for the K8s-hosted workloads, or do they stay rootful Docker-in-VM / containerd as K8s dictates?

References

• Narrative decision: Komodo over Kubernetes (K3s + ArgoCD) — the earlier conclusion this ADR re-examines.
• ADR 0001: Per-environment Git branches via Komodo ResourceSync — the merge-hygiene cost of Docker-native GitOps.
• ADR 0002: Baremetal Docker Host OS Selection
• ADR 0003: Container Runtime Tradeoffs

ADR: Skip local Docker registry, focus on git pipeline

Context

The homelab needs a strategy for managing container images between the local environment and production. Initial considerations included running a local container registry (Artifactory/Lightwell analog) or relying on Docker Hub with version pinning. The homelab already uses Komodo GitOps, GitHub Actions for rendering, and Ansible validation playbooks.

Goals:

1. Avoid infrastructure overhead — A local registry adds a running service, storage management, and maintenance burden for a feature (offline access) that isn’t actually required.
2. Leverage existing pipeline investment — GitHub Actions, Ansible validation, and Komodo ResourceSync are already in place and can be extended to cover image governance.
3. Security evaluation through PR process — Image scanning and review can happen in Git before anything gets deployed, rather than after images sit in a registry.
4. Version pinning is already the pattern — Komodo configs and Git branches already provide effective version control; a registry would be redundant.

Decision

Skip running a local Docker registry as a stop-gap. Instead:

1. Pin images explicitly in Komodo TOML configs (e.g., nginx:1.25-alpine instead of latest) — this is the primary guardrail.
2. Extend the existing Git pipeline to include:
   • Image scan on PR (Trivy or Anchore)
   • Push approved images to Docker Hub / existing registry with pinned tags
   • Komodo picks up the new commit SHA / tag on sync
3. Use Docker Hub rate-limit-friendly practices — pull once, cache locally via Docker’s built-in store, and pin versions.
4. Revisit local registry later only if offline access or rate limits become actual problems.

Considered alternatives

Consequences

Positive

• No extra running service or storage to manage
• Security gates move left — images are scanned and reviewed in PR before any deployment
• Existing Git/GitHub Actions investment leveraged; no new infra needed
• Version pinning already the pattern via Komodo configs; registry would be redundant
• Simpler onboarding for new services — just add to pipeline, no registry config needed

Negative / risks

• Image scanning depends on CI runner availability and speed
• If Docker Hub rate limits become problematic, no local cache (but this hasn’t been an issue in practice)
• Security evaluation relies on PR review quality — same as any code change

Mitigations

• Add Trivy scan to existing GitHub Action as a checklist step — fail PR if critical vulnerabilities found
• Cache scanned results in GitHub Actions artifact so subsequent scans are fast
• If rate limits bite, add a registry later as a targeted fix (not infrastructure upfront)
• Document the “pinning is the guardrail” principle so teams don’t revert to latest

Status

Accepted — proceed with extending the Git pipeline for image governance instead of running a local registry.

References

• Decisions: decisions.md
• Komodo GitOps: Komodo ResourceSync docs
• Existing ADR: 0001-komodo-resourcesync-branch-per-environment.md
• Trivy scanning: trivy image-scan --exit-code 1 --severity HIGH,CRITICAL

ADR: Baremetal Docker Host OS Selection

Context

The homelab needs a host OS for baremetal Docker nodes (LenovoMini 1 for Prod, LenovoMini 2 for Test, and the DevDocker VM on ProxMox). Requirements:

1. RPM/DNF-based — user preference, already familiar with the ecosystem.
2. Komodo GitOps compatibility — Periphery agent must be able to manage container workloads.
3. Low-maintenance — set once and forget; not something to constantly troubleshoot.
4. Easily automatable — host provisioning should be reproducible via Ansible, including Periphery agent setup.

Options Considered

Host OS

Container Runtime

Research Findings

Podman + Komodo Compatibility

After investigation, Podman support in Komodo relies on workarounds:

• Requires DOCKER_HOST env pointing at Podman’s Docker API socket (/run/user/$UID/podman/podman.sock)
• Stacks may incorrectly show as “DOWN” after successful deploy due to compose network label differences (GitHub issue #504)
• No first-class testing or documentation from Komodo maintainers
• Conclusion: Support is too rough for a low-maintenance setup

Rootful vs Rootless Decision

Rootless Docker was initially preferred for security, but three factors drove the switch to rootful:

• No practical benefit for a homelab — all workloads are user-deployed and trusted. The socket is already protected by host-level access controls (SSH, firewall, physical security).
• Avoids community-driven maintenance burden — rootless Docker with Komodo Periphery relies on community patterns not official docs. Rootful is the documented, tested path.
• Simplifies Ansible automation — rootful Docker is a single dnf install + daemon enable. No dockerd-rootless-setuptool.sh, no loginctl enable-linger, no environment variable wiring.

Host OS Alignment

• Fedora Server ships recent kernel + DNF, packages are fresh enough for both the Docker CE repo and Ansible management
• Rocky/Alma would work but are conservative; no advantage for this use case
• Fedora CoreOS is immutable — no package install at runtime, conflicts with Ansible-based provisioning

Decision (Provisional)

1. Fedora Server for the host OS — DNF-native, recent kernel, familiar tooling, easy Ansible provisioning.
2. Docker (not Podman) for the container runtime — Komodo’s first-class support removes any reason to fight Podman compatibility.
3. Rootful Docker — simplicity wins for a homelab with trusted workloads. No port restrictions, no privileged container limitations, no extra setup steps.
4. Containerized Periphery — deployed as a Docker container managed by Ansible, with /var/run/docker.sock mounted for API access. Updates via image pull, not manual binary replacement.

Implementation: Containerized Periphery via Ansible

Periphery runs as a Docker container, managed by the same Ansible playbook that provisions the host. The Komodo maintainer’s filesystem semantics concern (discussion #220) applies to rootless Docker where user namespaces cause path remapping — with rootful Docker, container and host paths are the same, so the concern doesn’t apply.

Ansible Role Structure

The host provisioning playbook will include a role komodo-periphery that handles Periphery as a container:

roles/
  komodo-periphery/
    tasks/
      main.yml          # pull, create dirs, run container
    templates/
      periphery.config.toml.j2   # config template
    vars/
      main.yml          # defaults (core URL, port, etc.)

Tasks (main.yml)

- name: Ensure Periphery data directory exists
  ansible.builtin.file:
    path: /opt/periphery
    state: directory
    mode: '0755'

- name: Deploy Periphery container
  community.docker.docker_container:
    name: periphery
    image: "ghcr.io/moghtech/periphery:{{ periphery_version }}"
    restart_policy: unless-stopped
    network_mode: host
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "/opt/periphery:/config"
      - "/:/mnt/host:ro"
    env:
      PERIPHERY_CONFIG_PATH: /config/periphery.config.toml

Key choices:

• network_mode: host — Periphery needs to reach containers by their host-level ports for monitoring. Host networking avoids port mapping conflicts and simplifies the network topology.
• /var/run/docker.sock mount — gives Periphery the standard Docker API access. With rootful Docker this is the native socket, no indirection needed.
• /:/mnt/host:ro — bind mounts the host root so Periphery can read filesystem stats for disk monitoring.
• restart_policy: unless-stopped — survives reboots and daemon restarts.

Config Template (periphery.config.toml.j2)

[periphery]
port = {{ periphery_port | default(8120) }}
root_directory = "/mnt/host"

[[connections]]
address = "wss://{{ core_address }}/connection"
connect_as = "{{ inventory_hostname }}"
onboarding_key = "{{ periphery_onboarding_key }}"

The config template is rendered by Ansible with host-specific variables (core address, hostname, onboarding key), sourced from Ansible vault or group vars.

Ansible Playbook Structure

- hosts: docker_hosts
  become: yes
  roles:
    - role: docker-install        # dnf install docker, enable + start
    - role: komodo-periphery      # pull image, deploy container

Gotchas

1. SELinux may block container socket access. On Fedora Server with SELinux enforcing, the Periphery container needs the :z flag on volume mounts or a dedicated SELinux policy. If Periphery fails to connect, check ausearch -m avc and add :z to the socket mount: "/var/run/docker.sock:/var/run/docker.sock:z".

2. Host networking limits to one Periphery per host. With network_mode: host, port 8120 is consumed directly on the host. Only one Periphery instance can run per machine. This is the expected topology — one agent per host.

3. Periphery container image tag strategy. Use a specific version tag (e.g., :0.2.5) not :latest for reproducible deployments. Update by bumping the version in Ansible vars and re-running the playbook.

4. Onboarding key rotates. The onboarding_key in [[connections]] is a one-time use key from the Komodo Core UI. After first connection, Periphery persists its session — the key is no longer needed. If re-deploying from scratch, generate a new key.

5. Config changes require container restart. If periphery.config.toml changes, Ansible must trigger a container restart. The docker_container module handles this automatically when config changes are detected via template diff.

6. Ansible vault for secrets. The onboarding key is sensitive and should be stored in Ansible vault, not in plaintext group vars.

Consequences

Positive:

• Familiar RPM/DNF management
• Komodo Periphery has native, first-class support for rootful Docker
• All workloads work without restriction — privileged containers, ports <1024, cgroup limits
• Containerized Periphery auto-updates via docker pull; Ansible handles version pinning
• Ansible-driven provisioning means new hosts are reproducible
• No special setup beyond standard Docker CE install

Negative / risks:

• Docker socket is root-equivalent — compromised container has host access. Mitigated by homelab trust model (all workloads are user-deployed).
• Containerized Periphery adds a Docker-in-Docker layer that the Komodo maintainer flags for potential filesystem confusion, but with rootful Docker (no user namespace) this is not a practical concern.
• Fedora ~13mo support window means OS upgrades every ~6-12mo (acceptable for homelab).
• Periphery config changes need container restart — automated via Ansible.

Open Questions

• Which Periphery image tag to pin initially? Start with :latest during bring-up, pin to a specific version once stable.
• How to handle Periphery version upgrades? Ansible var bump + playbook re-run. Need a process for checking upstream releases.

References

• ADR 0003: Container Runtime Tradeoffs — deeper analysis of Docker rootful, rootless, and Podman comparison
• FoxxMD — Migrating to Komodo (rootless Periphery via socket-proxy, systemd agent recommendation)
• Docker — Rootless Docker setup
• Komodo — Periphery installation docs
• Komodo issue #504 — Podman compose network status mismatch
• Komodo discussion #220 — Maintainer recommends systemd agent over containerized

ADR: Container Runtime Tradeoffs — Docker (Rootful), Docker (Rootless), and Podman

Context

Selecting a container runtime for baremetal Docker hosts involves balancing security, operational convenience, and tooling compatibility. This ADR breaks down three approaches and the fundamental tension between central-socket management and daemonless isolation.

The host OS decision is handled in ADR 0002: Baremetal Docker Host OS Selection.

Runtime Options

Docker (Rootful)

The classic model: dockerd runs as root, exposes /var/run/docker.sock.

Docker (Rootless)

dockerd runs under a user namespace, exposes a user-scoped socket.

Podman (Daemonless)

No central daemon. Each container is launched directly by the user’s Podman client.

Analysis

Security: The Central Socket Risk

A Docker socket is equivalent to root access on the host. Any process that can write to the socket can:

• Start containers with host filesystem mounts (read/write any file)
• Create privileged containers that break out of isolation
• Modify iptables, network namespaces, cgroups
• Access secrets mounted into any container

Rootful Docker places this power on a single, well-known path (/var/run/docker.sock). The docker group is effectively root. The blast radius of a compromised process with socket access is the entire host.

Rootless Docker mitigates this through user namespaces. The daemon and all containers run within the user’s namespace — root inside a container is the user outside. A compromised socket can’t escape the user namespace, so the blast radius is that user’s resources only. However, it’s still a central daemon — if the rootless dockerd process is exploited, all that user’s containers are compromised.

Podman eliminates the central socket entirely. No process management attachment, no daemon to target. Each container is a child of its launching process. The blast radius is per container — no socket to poison. To use management tooling, you must explicitly start a podman system service socket, which reintroduces a central endpoint — but it’s opt-in, not the default.

Management: Daemonless vs. Central Socket

This is the inverse of the security tradeoff:

Without a central socket, management tooling faces a discovery problem:

• How does Komodo’s Periphery find the container runtime? It can’t — it needs an explicit path to podman system service socket, which must be running as a systemd user service.
• How does docker ps / podman ps enumerate running containers across a session? Podman uses a per-user boltdb database, but there’s no persistent daemon registering container state.
• How do compose stacks get tracked? Podman’s compose support (podman-compose) uses a different project model than Docker Compose — Komodo’s stack status detection breaks (issue #504).

With a central socket, management is trivial: connect to the well-known path, issue standard API calls, get predictable results. The cost is the security risk described above.

Operational Considerations

Recommended Direction

This ADR does not make a standalone decision — it informs ADR 0002. The current direction:

Docker (rootful) for maximum simplicity and tooling compatibility. The socket-security risk is accepted as an acceptable tradeoff for a homelab with trusted workloads.

Podman is the more secure architecture by design, and rootless Docker is a middle ground. Rootful Docker optimizes for operational simplicity — no setup ceremony, no port restrictions, no privileged container limitations. For a homelab where all workloads are user-deployed and trusted, the central-socket risk is manageable.

If the homelab expands to include untrusted or multi-tenant workloads, this should be revisited — rootless Docker or Podman would contain blast radius in a way rootful Docker cannot.

Consequences

If Docker (rootful) is adopted (current direction):

• All management tooling works natively with zero configuration — standard Docker API on the well-known socket
• Setup is minimal — dnf install docker, enable daemon, run containers
• No workload restrictions — privileged containers, ports <1024, cgroup limits all work
• Socket is root-equivalent — any process with socket access owns the host. Acceptable for homelab trusted workloads.

If Docker (rootless) or Podman is reconsidered later:

• Stronger isolation — compromise is contained to a user namespace or individual container
• Some workloads (privileged containers, Swarm mode) are restricted
• Management complexity increases — socket discovery, env vars, emulation layers required for Komodo
• Ports <1024 need workarounds

References

• Docker: Rootless Docker
• Docker: Docker socket security
• Podman: Rootless Podman
• Podman: Systemd integration
• Komodo issue #504 — Podman compose network status mismatch
• ADR 0002: Baremetal Docker Host OS Selection

ADR: Per-environment Git branches via Komodo ResourceSync

Context

The homelab uses Komodo with ResourceSync to declare servers, stacks, deployments, and related resources from Git (an “app-of-apps” style flow: Git remains the audit trail; Komodo reconciles).

Goals:

1. main shows the whole picture — dev/, test/, and prod/ (or equivalent komodo/{dev,test,prod}/) layouts exist together on main so reviews and long-lived docs match reality after merges.
2. Per-environment steering — each environment’s sync can follow main or follow a named branch (for example to soak a stack change in Dev before it lands on main).
3. Homelab-realistic ergonomics — accept some manual merge hygiene instead of building full release automation up front.

An earlier sketch of this approach lived in wip/komodo-artifact-templates/scratch/branch_management_ideas.md.

Decision

1. Single infrastructure repo; main is the default line of truth
   After work merges, main holds the canonical layout for all managed environments (folder-separated configs, per existing repo structure docs).

2. One ResourceSync configuration per Komodo-managed environment (Dev, Test, Prod — or whichever environments are GitOps-managed)
   Each [[resource_sync]] (or equivalent Komodo sync configuration) sets, at minimum:

   • Repository (and git provider / account as required).
   • Branch (or other ref field supported by Komodo’s ResourceSync schema — see Resource sync config) independently per environment.
   • Resource path scope limited to that environment’s subtree (for example only komodo/dev/** for Dev), so a branch used for experiments does not accidentally apply Prod definitions.

3. Branch-testing workflow

   • Create a feature branch for the change under test.
   • Point the target environment’s ResourceSync at that branch (for example Dev follows feature/xyz while Test and Prod continue to follow main).
   • After validation, merge the feature branch to main.
   • Manually reconcile the ResourceSync metadata on main so environments that should track main again have their branch/ref set back to main (or another agreed default).

   Known quirk (accepted): the ResourceSync TOML that sets branch = "feature/xyz" lives in Git. On the feature branch that file typically must reference the same branch so the sync pulls a self-consistent tree. After merge to main, that same file’s branch field must be updated to main (or removed to default) for environments that should no longer track the feature branch. This is intentional small, explicit edits on main, not hidden automation, until/unless you add tooling.

4. Releases / tags as a pin for Prod (optional, not required now)
   Using tags or GitHub Releases so Prod tracks refs/tags/v… while main moves forward is a valid pattern when you want Prod immutability without branch gymnastics. With separate per-environment directories and per-sync branch control, the urgency is lower than in a single-shared-path model. We defer mandatory release-based promotion; revisit if main churn becomes painful for Prod.

Consequences

Positive

• Clear mental model: folders = promotion path; branch on ResourceSync = “which revision of that tree is live for this environment.”
• Feature work can soak on Dev/Test without forking the whole repo layout conceptually on main.
• Decisions stay in Git and stay reviewable.

Negative / risks

• Merge hygiene: forgetting to repoint a ResourceSync after merge can leave an environment tracking a stale or abandoned branch.
• Self-consistency: sync config and synced paths must agree on branch, or behavior is confusing during transitions.

Mitigations (recommended, non-blocking)

• Short checklist when merging infra PRs: “ResourceSync branches updated for Dev/Test/Prod as intended?”
• Keep ResourceSync definitions in small, dedicated files (for example under komodo/sync/ or similar) so diffs are obvious.
• Optionally add CI later (grep/assert expected branch for Prod) if mistakes recur.

Alternatives considered

References

• Komodo: Sync resources
• Homelab repo layout: Planned repository structure (includes promotion flow between environment folders)
• Prior scratch note: wip/komodo-artifact-templates/scratch/branch_management_ideas.md

ADR 0001: LlamaFarm as AI model management layer

Status: Proposed
Date: 2026-07-14

Context

The homelab has a dedicated AI VM running Ubuntu with assorted AI tools installed ad-hoc. There is no consistent model management layer — models are pulled manually, runtimes are configured individually (Ollama, vLLM, raw HuggingFace), and there is no unified API surface or RAG pipeline.

As part of a broader effort to make this VM “more than just a box with AI tools on it,” a management layer is needed for:

• Model lifecycle (download, serve, switch, retire)
• RAG pipeline (ingest documents, chunk, embed, query)
• Unified API surface (OpenAI-compatible endpoint for all models)
• Reproducible configuration (infra-as-code for AI, not just the VM)

Decision Drivers

1. Disposable VM model — the VM should be recreatable from Ansible + persistent data volumes without manual recovery steps.
2. Local-first, offline-capable — the AI layer must work without internet after models are cached.
3. GPU passthrough — quad-port P5000 needs CUDA support with minimal friction.
4. Config-driven, not script-driven — model selection, prompt templates, and RAG pipelines should live in version-controlled config files, not shell scripts.
5. OpenAI-compatible API — any tool that speaks OpenAI’s API should be able to target this VM.

Options Considered

Option 1: Ansible/Terraform/Semaphore only

Provision the VM, install Ollama/vLLM directly, manage models via shell scripts or ad-hoc commands.

• Pro: Mature tooling, full homelab coverage.
• Con: No model lifecycle abstraction — still need glue scripts for model switching, RAG pipelines, etc. Ansible does not natively understand “serve this model” as a primitive.

Option 2: LlamaFarm as the AI layer (chosen)

Provision the VM with Ansible, then run LlamaFarm natively (no containers) to manage models, RAG, and API serving.

• Pro: Config-driven (llamafarm.yaml), unified API, multi-model switching, built-in RAG, MCP support, no-container-required.
• Con: Pre-1.0 project (v0.0.34, YC W22), active development, small community.

Option 3: Kubernetes + Kubeflow / KServe

Run a lightweight K8s cluster on the VM, deploy models via KServe or similar.

• Pro: Production-grade scaling, model canarying.
• Con: Massive operational overhead for a single-VM homelab. K8s on a single node adds complexity with zero benefit at this scale.

Option 4: Docker Compose stack (Ollama + Open WebUI + Qdrant + custom API wrapper)

Orchestrate individual OSS components via Compose.

• Pro: Each component is mature individually.
• Con: Glue code for integration, more surface area to maintain, no single config file.

Decision

Adopt LlamaFarm as the AI model management layer, running natively (no containers) on a Debian VM with Ansible for infrastructure provisioning.

Ansible handles:

• OS hardening, NVIDIA driver + CUDA toolkit installation
• Persistent storage mount points
• LlamaFarm CLI installation (lf) via curl | bash
• $LF_DATA_DIR and $TRANSFORMERS_CACHE redirection to persistent volumes
• systemd unit for LlamaFarm services
• Reverse proxy (nginx) with TLS termination

LlamaFarm handles:

• Model downloading, caching, and serving
• Multi-model configuration (fast vs powerful, local vs remote)
• RAG pipeline (document parsing, embedding, retrieval)
• OpenAI-compatible REST API
• MCP tool integration

Consequences

Positive

• Single config file (llamafarm.yaml) is the source of truth for all AI behavior — model selection, prompts, RAG, MCP servers.
• Truly disposable VM — models and data live on persistent volumes; a fresh Debian install + one lf init + lf start + pointing at cached models restores full capability.
• Built-in RAG — no need to wire up a separate vector DB, embedder, and chunker; it’s all in the config.
• Native deployment — LlamaFarm runs without Docker/K8s, keeping the footprint lean and the GPU path simple.
• Offline-capable — after initial model download, all inference is local with no egress.

Negative / Risks

Mitigations

References

• LlamaFarm docs: https://docs.llamafarm.dev
• GitHub: https://github.com/llama-farm/llamafarm (Apache 2.0, 833 stars, v0.0.34)
• AI VM architecture: docs/homelab/ai-vm/architecture.md

ADR: Home Assistant Deployment

Date: 2026-05-11
Status: Accepted
Deciders: Self

Context

Home Assistant (HA) is the central hub for IoT automation in the new house. It needs to integrate with:

• Zigbee devices (switches, sensors, buttons) via USB coordinator
• Tapo cameras (RTSP/Tapo integration)
• Unifi Protect (running on UDM)
• Google Home (TTS announcements)
• Sensi thermostats (cloud API)

The homelab has a defined Prod/Test/Dev architecture with Komodo-managed Docker hosts. Home Assistant must fit into this architecture while meeting its unique requirements.

Requirements

1. USB access — Zigbee coordinator must be physically attached to host
2. Network access — Must reach IoT VLAN (10.0.4.0/24) for cameras, Google Home, etc.
3. Unifi Protect access — Must reach UDM for Protect integration
4. Reliability — Automation failures directly impact daily life
5. Manageability — Should fit into existing GitOps/Komodo workflow where possible
6. Backup/Recovery — Must be recoverable quickly if host fails

Constraints

• IoT VLAN (10.0.4.0/24) is isolated from other VLANs by design
• Zigbee coordinator is USB-bound — cannot easily fail over to another host
• Home Assistant does not support native clustering/HA

Decision

Deploy Home Assistant as a container on LenovoMini 1 (Prod Docker host) with USB passthrough for the Zigbee coordinator.

Phased Approach

Phase 1 — Bootstrap (RPi):

• Run HA on existing Raspberry Pi 4
• Place on Prod VLAN (10.0.1.0/24)
• Configure firewall rules for IoT access
• Validates the setup before full integration

Phase 2 — Production Migration:

• Migrate HA to container on LenovoMini 1
• USB passthrough for Zigbee coordinator
• Komodo-managed deployment
• Automated backups to UnRaid NAS

Options Considered

Option 1: Dedicated Raspberry Pi

Description: Run Home Assistant OS on a dedicated RPi 4.

Pros:

• Simple setup
• Direct USB access (no passthrough complexity)
• Isolated from other workloads

Cons:

• Another device to manage outside Komodo
• SD card reliability concerns
• Limited compute for future expansion (Frigate, etc.)
• No integration with existing backup infrastructure

Verdict: Good for bootstrap, not ideal long-term.

Option 2: Container on Prod Docker Host (LenovoMini 1)

Description: Run HA as a Docker container on the existing Prod host, managed via Komodo.

Pros:

• Better hardware (8th Gen i7, 16GB RAM, NVIDIA GPU)
• Integrated with existing management (Komodo, GitOps)
• Backups to UnRaid NAS
• Network already on Prod VLAN with access to other services
• GPU available for future Frigate integration

Cons:

• USB passthrough required (adds configuration)
• Shares resources with Plex/*arr (mitigated by resource limits)
• Single point of failure (mitigated by backups)

Verdict: Best fit for long-term architecture.

Option 3: VM on ProxMox

Description: Run HA in a dedicated VM on the ProxMox host.

Pros:

• Isolated resources
• ProxMox has snapshot/backup features

Cons:

• ProxMox is on Dev VLAN (10.0.3.0/24) — wrong network segment
• USB passthrough to VM is more complex than Docker
• Not on the Komodo-managed path

Verdict: Network placement is wrong. Rejected.

Option 4: Dedicated Mini Host on Prod VLAN

Description: Purchase a dedicated small device (NUC, mini PC) for HA.

Pros:

• Dedicated to home automation
• Direct USB, proper network placement

Cons:

• Additional hardware cost
• Another device outside existing management
• Overkill given available capacity on LenovoMini 1

Verdict: Unnecessary given existing infrastructure.

Consequences

Positive

• Integrated management: HA deployment is Komodo-managed, follows GitOps patterns
• Better hardware: More headroom for future expansion (Frigate, more integrations)
• Unified backups: HA config/data backed up to UnRaid alongside other Prod data
• Network positioning: Prod VLAN has (or will have) appropriate access to IoT and UDM

Negative

• USB passthrough complexity: Requires privileged container or explicit device mapping
• Shared host: HA competes with Plex/*arr for resources (mitigated with resource limits)
• Single point of failure: If LenovoMini 1 goes down, HA is down (mitigated by backups + quick restore)

Neutral

• No native HA failover: Accepted limitation — most homelabs operate this way
• Zigbee bound to host: Coordinator cannot fail over, but Zigbee direct bindings provide local fallback

Implementation

Phase 1: Bootstrap on RPi

Timeline: Immediate (while homelab is being set up)

1. Install Home Assistant OS on Raspberry Pi 4
2. Connect Zigbee coordinator (Sonoff dongle)
3. Place RPi on Prod VLAN (10.0.1.0/24)
4. Configure firewall rules:
   • Allow Prod → IoT (for cameras, Google Home)
   • Allow Prod → UDM management interface (for Unifi Protect)
5. Set up integrations (Zigbee, Tapo, Protect, Google Cast, Sensi)
6. Configure automations
7. Validate everything works

Backup: Manual snapshots to USB or network share

Phase 2: Migrate to Prod Docker

Timeline: When homelab Prod environment is stable

1. Prepare LenovoMini 1:

   • Identify USB port for Zigbee coordinator
   • Test USB device passthrough (/dev/ttyUSB0 or /dev/ttyACM0)

2. Create HA container configuration:

   # docker-compose.yml (Komodo-managed)
   services:
     homeassistant:
       container_name: homeassistant
       image: ghcr.io/home-assistant/home-assistant:stable
       restart: unless-stopped
       privileged: true  # Required for USB and network discovery
       network_mode: host  # Simplifies discovery protocols
       volumes:
         - /opt/homeassistant/config:/config
         - /etc/localtime:/etc/localtime:ro
       devices:
         - /dev/ttyUSB0:/dev/ttyUSB0  # Zigbee coordinator
       environment:
         - TZ=America/New_York

   Notes:

   • privileged: true or explicit device mapping required for USB
   • network_mode: host recommended for mDNS/SSDP discovery (Google Cast, etc.)
   • Adjust device path based on actual Zigbee dongle

3. Migrate configuration:

   • Create HA snapshot on RPi
   • Restore snapshot to container config volume
   • Verify Zigbee coordinator is detected
   • Verify all integrations reconnect

4. Set up backups:

   • Mount UnRaid NAS share for backup storage
   • Configure HA to snapshot to NAS (daily)
   • Or use Komodo/external script to backup /opt/homeassistant/config

5. Add to Komodo:

   • Create TOML resource for HA container
   • Enable GitOps management
   • Set deploy: false initially (manual approval for HA changes)

6. Decommission RPi:

   • Verify Prod deployment is stable (run parallel for a few days if desired)
   • Power down RPi
   • Keep as cold spare for emergency restore

Network Configuration

Firewall rules needed (on UDM):

Note: IoT VLAN should remain isolated from other VLANs. Only Prod (where HA lives) gets access.

Backup & Recovery

Backup strategy:

• Daily HA snapshots to UnRaid NAS
• Retain 7 daily + 4 weekly snapshots
• Config directory also backed up via standard Prod backup job

Recovery procedure:

1. If LenovoMini 1 fails:
   • Option A: Restore to LenovoMini 2 (Test) temporarily
   • Option B: Boot RPi with latest snapshot
2. Restore latest snapshot
3. Reconnect Zigbee coordinator (may need re-pairing if coordinator changed)
4. Verify automations

RTO target: < 1 hour (acceptable for home automation)

Resilience Notes

What survives HA downtime:

• Zigbee direct bindings (button → switch) work without HA
• Smart switches work manually (physical toggle)
• Cameras continue recording (Protect is independent)
• Thermostats continue on last schedule (Sensi is independent)

What fails:

• Automations (door alerts, scenes, presence-based lighting)
• Voice control via HA (Google Home native commands still work)
• Unified dashboard/control

Acceptable trade-off: Brief outages are tolerable. Critical functions (lights, climate) have manual fallback.

Phase 3: Automated Failover (Stretch Goal)

Status: Future consideration
Complexity: Medium-high
Additional hardware: Second Zigbee coordinator (~$25)

Concept

Implement a warm/cold spare HA instance on the Test environment (LenovoMini 2) with automated failover capability. When the primary (Prod) instance has unfixable issues, a single-click remediation brings up the spare and updates network rules.

Architecture

                     Normal Operation
┌─────────────────────────────────────────────────────────────┐
│                                                             │
│   Prod VLAN (10.0.1.0/24)              IoT VLAN (10.0.4.0/24)
│   ┌─────────────────────┐              ┌─────────────────┐  │
│   │   LenovoMini 1      │   allowed    │  Zigbee devices │  │
│   │   [HA - Active]     │─────────────▶│  Cameras        │  │
│   │   [Zigbee Coord 1]  │              │  Google Home    │  │
│   └─────────────────────┘              └─────────────────┘  │
│                                                 ▲           │
│   Test VLAN (10.0.2.0/24)               blocked │           │
│   ┌─────────────────────┐                       │           │
│   │   LenovoMini 2      │───────────────────────┘           │
│   │   [HA - Standby]    │                                   │
│   │   [Zigbee Coord 2]  │                                   │
│   └─────────────────────┘                                   │
│                                                             │
└─────────────────────────────────────────────────────────────┘

                     Failover State
┌─────────────────────────────────────────────────────────────┐
│                                                             │
│   Prod VLAN (10.0.1.0/24)              IoT VLAN (10.0.4.0/24)
│   ┌─────────────────────┐              ┌─────────────────┐  │
│   │   LenovoMini 1      │   blocked    │  Zigbee devices │  │
│   │   [HA - DOWN]       │──────────X   │  Cameras        │  │
│   │                     │              │  Google Home    │  │
│   └─────────────────────┘              └─────────────────┘  │
│                                                 ▲           │
│   Test VLAN (10.0.2.0/24)               allowed │           │
│   ┌─────────────────────┐                       │           │
│   │   LenovoMini 2      │───────────────────────┘           │
│   │   [HA - Active]     │                                   │
│   │   [Zigbee Coord 2]  │                                   │
│   └─────────────────────┘                                   │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Components

1. Standby HA Instance (LenovoMini 2)

• Home Assistant container (same config as Prod)
• Second Zigbee coordinator (Sonoff dongle)
• Container stopped or running in standby mode

2. Config Synchronization

• Option A: Scheduled rsync of /opt/homeassistant/config from Prod → Test (via NAS or direct)
• Option B: Shared config on NAS mount (both read, only active writes)
• Option C: Git-based config with HA Git integration

3. Zigbee Network Portability

• Use Zigbee2MQTT (not ZHA) — stores network config in coordinator_backup.json
• Backup can be restored to second coordinator
• Devices don’t need re-pairing if network key is preserved
• Caveat: Second coordinator must be same model or compatible

4. Firewall Automation (Ansible/Terraform)

# ansible playbook: failover-to-test.yml
- name: Failover HA to Test environment
  hosts: localhost
  tasks:
    - name: Update UDM firewall - block Prod to IoT
      community.general.udm_firewall_rule:
        name: "Prod-to-IoT"
        state: disabled
      
    - name: Update UDM firewall - allow Test to IoT
      community.general.udm_firewall_rule:
        name: "Test-to-IoT"
        state: enabled
        
    - name: Start HA container on Test
      community.docker.docker_container:
        name: homeassistant
        state: started
      delegate_to: lenovomini2
      
    - name: Stop HA container on Prod (if reachable)
      community.docker.docker_container:
        name: homeassistant
        state: stopped
      delegate_to: lenovomini1
      ignore_errors: yes

5. Failback Procedure

• Reverse the process when Prod is healthy
• Sync any config changes from Test back to Prod
• Re-enable Prod firewall rules, disable Test

Spare Modes

Recommendation: Start with Cold Spare. Warm Spare adds complexity for marginal time savings.

Triggering Failover

Manual (Phase 3a):

• Single-click: Run Ansible playbook from Komodo UI or CLI
• Human detects issue, initiates failover

Automated (Phase 3b, future):

• Health check monitors HA (e.g., /api/ endpoint)
• Consecutive failures trigger alert
• Optional: Auto-failover after N failures (risky, needs tuning)

Prerequisites

Before implementing:

1. Hardware: Second Zigbee coordinator (~$25)
2. Network: Test VLAN firewall rules pre-configured (disabled)
3. Ansible: Playbooks for failover/failback
4. UDM API access: Ansible collection for Unifi or direct API calls
5. Zigbee2MQTT: Migrate from ZHA if using ZHA initially
6. Testing: Validate failover in controlled scenario

Risks & Mitigations

Cost

Decision

Defer to Phase 3. Implement after Phase 2 is stable and proven. The complexity is justified only after experiencing reliability issues or when uptime requirements increase.

References

• Home Assistant Container Installation
• Zigbee2MQTT Docker
• Zigbee2MQTT Coordinator Backup
• Home Assistant Backup/Restore
• Unifi API

Decision: Door Locks

Date: 2026-05-11
Status: Deferred
Decision: No smart locks initially; rely on physical keys + camera detection

Context

Evaluating whether to include smart locks in the initial IoT deployment.

Decision

Defer smart locks. Focus on detection, not prevention.

Rationale

1. Security philosophy: If someone is determined to get in, they will. Detection and evidence (cameras) are more valuable than a smart lock.

2. Existing coverage: Unifi Protect cameras + door sensors provide detection and alerting.

3. Complexity trade-off: Smart locks add battery management, failure modes, and integration points without a clear use case.

4. Physical keys work: No current pain point that a smart lock solves.

Future Considerations

Revisit when one of these use cases becomes valuable:

• Keyless entry for family
• Temporary access codes for guests/contractors
• Auto-lock after X minutes
• Remote verification
• Presence-based auto-unlock

Options to Evaluate Later

Consumer (HA compatible):

• Yale Assure Lock 2 (Zigbee, Z-Wave, Matter)
• Schlage Encode Plus (WiFi, HomeKit)
• August Wi-Fi Smart Lock (retrofit)
• Level Lock+ (invisible, HomeKit/Matter)

Enterprise:

• Unifi Access — NFC/card based, integrates with Protect
• Higher cost (~$300+/lock + hub), more commercial-grade
• Interesting for “set and forget” but overkill for residential

When Adding Locks

1. Prefer Zigbee or Matter (aligns with existing protocol)
2. Integrate via Home Assistant
3. Automate: auto-lock, lock-on-leave, alert if unlocked while away
4. Tie to camera snapshots on unlock events

Decision: Smart Switch Protocol

Date: 2026-05-11
Status: Decided
Decision: Zigbee (Inovelli Blue switches)

Context

Planning IoT lighting for new house. Requirements:

• Work in 3-way configurations with only ONE switch replaced
• Support wireless buttons for remote control
• Prioritize cost savings and upgradeability
• Integrate with self-hosted Home Assistant

Neutral wire is present.

Options Considered

Zigbee — Inovelli Blue Series

Protocol: Zigbee 3.0
Switch Cost: ~$45-50
Coordinator Cost: ~$20-30

Pros:

• Works with existing dumb switch in 3-way config
• Large device ecosystem (sensors, buttons, plugs)
• Cheap wireless buttons ($7-15)
• Direct Zigbee binding (button controls switch without hub)
• Configurable LED notification bar
• Scene/multi-tap support
• Mesh network extends coverage

Cons:

• Higher per-switch cost
• 2.4GHz interference possible (manageable)

Z-Wave — Zooz Switches

Protocol: Z-Wave Plus
Switch Cost: ~$30-35
Coordinator Cost: ~$35-45

Pros:

• Works with existing dumb switch in 3-way config
• Lower per-switch cost
• Dedicated frequency (no WiFi interference)
• Mature, reliable protocol

Cons:

• Smaller device ecosystem
• Wireless buttons limited and expensive ($40+)
• Less feature-rich

Also Considered

Lutron Caseta: Excellent reliability, but requires proprietary bridge. Less flexible ecosystem.

WiFi switches: Cloud dependency or complex local setup. Doesn’t align with self-hosted goals.

Decision

Zigbee with Inovelli Blue switches.

Rationale

1. Accessory ecosystem: Zigbee buttons and sensors are significantly cheaper. $7 IKEA button vs $40+ Z-Wave compounds quickly.

2. Upgradeability: Largest device selection across manufacturers for future expansion.

3. Feature set: LED notifications, direct binding, scene control — useful for automations.

4. 3-way handling: Single switch replacement works cleanly.

Implementation

• Coordinator: Sonoff Zigbee 3.0 USB Dongle Plus
• Integration: ZHA or Zigbee2MQTT (both well-supported)
• Placement: Central location or USB extension to avoid interference

Home Assistant Automations

Workflow definitions for IoT automations. Copy/adapt these for Home Assistant.

Door Sensor Automations

Door Left Open Alert

Announces via Google Home when a door has been open for 5 minutes.

alias: "Alert - Door Left Open"
description: "Announce when a door has been open too long"
trigger:
  - platform: state
    entity_id:
      - binary_sensor.front_door
      - binary_sensor.back_door
      - binary_sensor.office_door
      - binary_sensor.garage_entry_door
    to: "on"
    for:
      minutes: 5
action:
  - service: tts.google_translate_say
    target:
      entity_id: media_player.living_room_speaker
    data:
      message: "{{ trigger.to_state.attributes.friendly_name }} has been open for 5 minutes."
mode: parallel

Door Still Open Reminder

Repeats announcement every 5 minutes while an exterior door remains open.

alias: "Alert - Door Still Open Reminder"
description: "Repeat reminder every 5 minutes while door is open"
trigger:
  - platform: state
    entity_id:
      - binary_sensor.front_door
      - binary_sensor.back_door
    to: "on"
    for:
      minutes: 5
action:
  - repeat:
      while:
        - condition: state
          entity_id: "{{ trigger.entity_id }}"
          state: "on"
      sequence:
        - service: tts.google_translate_say
          target:
            entity_id: media_player.living_room_speaker
          data:
            message: "Reminder: {{ trigger.to_state.attributes.friendly_name }} is still open."
        - delay:
            minutes: 5
mode: parallel

Garage Entry Night Alert

Alerts when garage entry door opens at night, with camera snapshot.

alias: "Alert - Garage Entry Night"
description: "Alert when garage entry opens at night"
trigger:
  - platform: state
    entity_id: binary_sensor.garage_entry_door
    to: "on"
condition:
  - condition: time
    after: "22:00:00"
    before: "06:00:00"
action:
  - service: tts.google_translate_say
    target:
      entity_id: media_player.bedroom_speaker
    data:
      message: "Garage entry door opened."
  - service: camera.snapshot
    target:
      entity_id: camera.tapo_garage
    data:
      filename: "/config/www/snapshots/garage_{{ now().strftime('%Y%m%d_%H%M%S') }}.jpg"
mode: single

Lighting Automations

Garage Light Button

Wireless button in adjacent room toggles garage light.

alias: "Button - Garage Light Toggle"
description: "Wireless button controls garage light"
trigger:
  - platform: event
    event_type: zha_event
    event_data:
      device_id: <zigbee_button_device_id>
      command: "toggle"
action:
  - service: light.toggle
    target:
      entity_id: light.garage
mode: single

Note: Can also use Zigbee direct binding for button-to-switch control without HA in the loop.

Goodnight Scene

Single button or voice command triggers bedtime routine.

alias: "Scene - Goodnight"
description: "Turn off lights, set fan, dim bedroom"
trigger:
  - platform: event
    event_type: zha_event
    event_data:
      device_id: <bedside_button_device_id>
      command: "on"
action:
  - service: light.turn_off
    target:
      entity_id:
        - light.hall_light
        - light.garage
  - service: light.turn_on
    target:
      entity_id: light.bedroom_fan_light
    data:
      brightness_pct: 10
  - service: fan.set_percentage
    target:
      entity_id: fan.bedroom_fan
    data:
      percentage: 50
mode: single

Presence Automations

Arriving Home

Turns on entry lights when arriving home after sunset.

alias: "Presence - Arriving Home"
description: "Turn on lights when someone arrives after sunset"
trigger:
  - platform: state
    entity_id: person.you
    to: "home"
condition:
  - condition: sun
    after: sunset
action:
  - service: light.turn_on
    target:
      entity_id:
        - light.front_porch
        - light.living_room
mode: single

Presence detection options:

• HA Companion app (phone GPS)
• Unifi device tracker (via Unifi integration)
• Bluetooth/BLE presence sensors (future)

Climate Automations (Future)

With Sensi thermostats integrated, potential automations:

Away Mode

alias: "Climate - Away Mode"
description: "Set thermostats to away when everyone leaves"
trigger:
  - platform: state
    entity_id: group.all_people
    to: "not_home"
    for:
      minutes: 30
action:
  - service: climate.set_preset_mode
    target:
      entity_id:
        - climate.sensi_upstairs
        - climate.sensi_downstairs
    data:
      preset_mode: "away"
mode: single

Pre-condition Before Arrival

alias: "Climate - Pre-condition"
description: "Resume normal temps when someone is heading home"
trigger:
  - platform: zone
    entity_id: person.you
    zone: zone.near_home
    event: enter
action:
  - service: climate.set_preset_mode
    target:
      entity_id:
        - climate.sensi_upstairs
        - climate.sensi_downstairs
    data:
      preset_mode: "home"
mode: single

Integration Notes

Google Cast TTS

Uses tts.google_translate_say service. Requires:

• Google Cast integration configured in HA
• Speaker entity IDs (e.g., media_player.living_room_speaker)

Tapo Camera Snapshots

Requires Tapo integration or generic camera with RTSP. Snapshots save to /config/www/ for web access.

Zigbee Events

Button events use zha_event (for ZHA) or mqtt triggers (for Zigbee2MQTT). Device IDs and commands vary by button model — check HA device page after pairing.

Receipt Automation

Receipt Fridge Launch — NFC App Launch

Tap the fridge NFC tag to open the receipt scanner app directly.

alias: "Receipts - NFC App Launch"
description: "Launch receipt scanner app from fridge NFC"
trigger:
  - platform: nfc
    tag_id: <fridge_receipt_nfc_uid>
action:
  - service: notify.mobile_app_<phone>
    data:
      message: "open_receipt_scanner"
      data:
        action: launch_app
        package: com.example.receiptscanner
mode: single

The NFC tag is the entry point. The app handles the actual capture and any per-receipt workflow.

Considerations:

• If the app supports deep links or URL schemes, prefer that over a generic notification.
• If people use different phones/scanner apps, map multiple notify targets or a choose block by person/presence.

Service Check / Router Status NFC

Router Status — NFC Status Page

Opens a locally hosted status page showing service health.

alias: "Status - Router NFC"
description: "Serve local status page for router/service health"
trigger:
  - platform: nfc
    tag_id: <router_status_nfc_uid>
action:
  - service: browser_mod.popup
    data:
      title: "Service Status"
      # Serve from a local/static endpoint; no external CDN deps
      url: "/local/status/index.html"
mode: single

Implementation notes:

• Host status/index.html on the same host that serves HA or a lightweight internal web server.
• Keep the page plain HTML with no external dependencies so it works when the internet is down.
• Display gateway ping, DNS resolution result, Pi-hole stats, and a one-click router reboot if supported.

Consumable Replacement NFC

Consumable Tag — Reorder Trigger

Opens the replacement item’s reorder path.

alias: "Consumables - Replace"
description: "Open consumable replacement action"
trigger:
  - platform: nfc
    tag_id: <consumable_nfc_uid>
action:
  - service: browser_mod.popup
    data:
      title: "Replace Air Filter"
      # Prefer a stable local/HA endpoint; the destination can rotate behind this URL
      url: "/local/consumables/air-filter.html"
mode: single

Instead of encoding a retailer URL directly, use a stable local or Home Assistant destination so the reorder page can change without rewriting the tag.

Pi-hole Temporary Disable NFC

Pi-hole — 30 Minute Disable

Disables Pi-hole time-boxed for 30 minutes.

alias: "Pi-hole - Temp Disable 30m"
description: "Disable Pi-hole for 30 minutes from NFC tap"
trigger:
  - platform: nfc
    tag_id: <pihole_temp_disable_nfc_uid>
action:
  - service: rest_command.pihole_disable_30m
mode: single

Time-box the disable window so ad-blocking is re-enabled automatically. Surface the remaining time on the local status page so the household can see the countdown.

Public status site and operator dashboard

Plans for user-facing public status (status.samplayskeys.com) and a private operator “single pane of glass,” with acceptance criteria, sequencing, and open decisions.

Phase 1 — Public status site

URL: status.samplayskeys.com
Audience: Partner and other users of user-facing services (not homelab admins).
Tone / UX: Similar to GitHub Status: components (or service groups), overall status, recent incident history, minimal jargon.

Intent

• User-available observability: clear “is the thing I use up?” without exposing internal topology or admin tools.
• Survives local outage: host on a VPS (candidate: Oracle Cloud free tier) so the page stays reachable if home internet or the lab is down.

Success criteria

1. Public HTTPS at status.samplayskeys.com with a sensible TLS story (e.g. reverse proxy + Let’s Encrypt on the VPS).
2. Lists only non-admin, user-relevant resources (e.g. Plex, Signal, Miniflux, Syncthing client-facing checks — exact list is yours to curate).
3. Does not link to or name internal admin URLs, VLANs, or management UIs.
4. Status data remains trustworthy when the house is offline: checks must run from the VPS (or another off-LAN path) against reachable endpoints (public hostname, reverse proxy, or approved tunnel — not 10.x.x.x).
5. Optional but valuable: brief incident log / maintenance notices you can edit without redeploying the whole stack (even a static markdown or small API is fine).

Open decisions (Phase 1)

Non-goals (Phase 1)

• Full metrics graphs, Prometheus, or environmental sensors (already deferred elsewhere).
• Authenticated “admin view” of the same page (keep public surface boring and safe).

Phase 2 — Single pane of glass (operator)

Audience: You (operator), not the general public.
Primary job: One place to orient and jump to deeper tools.

Intent

1. Overview of the entire lab — health/signal at a glance (CPU/load/disk on key hosts, maybe Docker summary), possibly via Glances or equivalent on hosts plus an aggregation story.
2. Curated links (always visible):
   • Uptime Kuma (alerting / synthetics).
   • Komodo (deployments / stacks).
3. Dynamic or data-driven links to management URLs: Unraid UI, container admin UIs, UniFi, Synology DSM, etc. — ideally driven from the same source of truth you use for Komodo or a small YAML manifest so new services do not require hand-editing HTML.

Success criteria

1. One bookmarkable URL (likely not public — Tailscale or LAN-only) that opens the dashboard.
2. Sections or tiles: “Lab overview” (metrics snapshot), “Monitoring”, “Orchestration”, “Management consoles” (grouped by host or role).
3. Every critical jump target reachable from that page in ≤2 clicks without remembering IPs.
4. Link list can be updated without fragile one-off edits (manifest, template, or generator).

Open decisions (Phase 2)

Relationship to Phase 1

Suggested sequencing

1. Inventory user-facing hostnames and which are probe-able from the public internet (required for Phase 1).
2. Stand up VPS + DNS + TLS for status.samplayskeys.com.
3. Implement Phase 1 status page and curate components.
4. Phase 2: choose portal + Glances (or alternative) layout, add manifest for management links, wire Komodo/Kuma bookmarks.

Pre-mortem hooks (when you pick tools)

• Status on VPS: “Checks pass from Oracle but users still can’t reach the app” — split your connectivity vs user path (DNS, CDN, home IP change).
• Operator dashboard: “I pasted every admin URL into git” — rotation and leak risk; prefer Tailscale MagicDNS names + no secrets in repo.

When you name concrete software for Phase 1 or 2, run a short Pre-Mortem before locking in.