Appearance
Troubleshooting
Common failures and their fixes, grouped by component. Each component also ships a built-in diagnostic — start there:
sh
burrowee doctor # cli
burrowee gateway doctor # gateway
burrowee edge doctor # edgestatus is the read-only alias of doctor everywhere — the same report, never remediates, always exit 0. doctor --fix applies the thin remediations it can (e.g. starting a down daemon); --yes skips the prompts. The edge's doctor additionally exits 3 when any health row shows ✗ (a red edge and a broken CLI are different facts).
Every binary follows one exit contract: explicit --help (or a bare parent verb) prints that level's help page on stdout at exit 0; any malformed invocation — unknown verb, bad flag, stray argument — exits 2 with the right level's page on stderr; a command that ran and failed exits 1. Every stderr line the daemons write is timestamped, so the log files below are diagnosable after the fact.
Install (all components)
| Symptom | Cause | Fix |
|---|---|---|
| macOS Gatekeeper blocks a downloaded binary | The quarantine attribute on a manually downloaded file (the signed installer flow doesn't hit this) | xattr -d com.apple.quarantine ./burrowee-cli (repeat per binary) |
| Installer aborts: minisign missing | minisign is the trust root and is never auto-fetched | Install it from your package manager (brew install minisign / apt-get install minisign) and re-run |
burrowee: command not found right after installing the gateway or edge | The binaries are in the exec root /usr/local/burrowee/bin, which is on nobody's PATH — the installer prints the line that adds it and never applies it | Run the two lines from the installer's Next steps block (reproduced per shell in Put the exec root on PATH), or use the full path: /usr/local/burrowee/bin/burrowee gateway status. Not hash -r — the directory was never on PATH, so there is nothing cached to clear |
burrowee <component> → "is not installed on this machine" (exit 127) | The dispatcher found no burrowee-<component> (or, for gateway/edge, its -cli sibling) — gateway, edge, relay, and register are resolved at /usr/local/burrowee/bin only, by absolute path; cli, agent, and console are searched on PATH only | Install the missing component from Install; for a per-user component check ~/.local/bin is on PATH. The report names which rule applied and where it looked, and the installed here: … line lists which components are installed here, so you can tell a missing install from a PATH problem |
| Right after an install/update, the shell still runs an old version | The installer swept a stale copy that shadowed the fresh install — a pre-0.2.0 per-user binary, or a 0.2-era binary or symlink left in /usr/local/bin — but your shell cached the old path | hash -r (or open a new shell) so the shell re-resolves the command |
Updates & versions (all components)
Every component's status/doctor/--version output prints a versions block showing the binary installed on disk next to the version actually running as a service (including a row for the component's updater agent).
| Symptom | Cause | Fix |
|---|---|---|
Versions block shows "⚠ drift" (installed vX · running vY) | An update replaced (or staged) the binary on disk but the running service hasn't restarted onto it yet — normal after a staged edge update, update --no-restart, or briefly during a console-pushed update | burrowee restart / burrowee gateway restart / burrowee edge restart restarts the affected daemon onto the installed binary. See CLI updates, Gateway service, Edge operations |
An updater agent (burrowee-gateway-updater / burrowee-edge-updater) shows drift | update deliberately never touches the updater's own binary — that's what updater upgrade is for | burrowee gateway updater upgrade / burrowee edge updater upgrade advances the updater itself; burrowee <component> updater restart restarts its agent |
| The version reads current, but an older release's state migration clearly never ran (e.g. a gateway/edge still on pre-0.2.0 per-user paths while reporting 0.2.x) | The installer's migration gate trusts the recorded version and compares only MAJOR.MINOR.PATCH — hand-placed binaries, a missing/wrong version anchor, or a same-version rebuild leave the host looking "already migrated" | The hosted upgrade one-liner re-installs and force-runs the release's migrations, ignoring the recorded version — name the floor of the work that was skipped: curl -fsSL https://release.burrowee.com/<component>/upgrade.sh | sh -s -- 0.2.0. See Upgrading |
| Console shows an older version than what's actually running | The console mirrors the last state the component reported; a very recent restart hasn't reported yet | Give it a moment, or check the authoritative local state with burrowee <component> status |
update --dry reports a version gap but nothing changes | --dry only prints the installed-vs-available gap — it never installs | Re-run without --dry. On an edge, update stages the new binary by default (it takes effect on the next restart) — --auto applies and restarts immediately |
An update ran but the version didn't move backwards as expected | Updates are verified end-to-end and a resolved update target can never silently downgrade a host | By design. To deliberately roll back, pin the exact version: update --version <version> |
| A console push-update is refused / never lands on a gateway or edge | Push needs both the opt-in (allow_push_update) and a connected standalone updater agent — a node can be online with push enabled yet still refuse if its updater agent isn't running | The console now shows the refusal reason next to the Update button. Locally: burrowee gateway updater doctor / burrowee edge updater doctor (--fix starts the agent); check the opt-in with push status. On an edge, push allow also (re)starts the agent |
CLI
| Symptom | Cause | Fix |
|---|---|---|
bootstrap hangs at "waiting for approval…" then times out | Approval must come from the gateway's local console on the gateway host — not console.burrowee.com | Have the gateway operator check the local console for the pending pairing request and approve it. If it never appeared, the blob may reference the wrong relay — ask for a fresh blob |
| "decrypt blob (wrong PIN?)" | The PIN doesn't match the blob | Ask the gateway operator to confirm the PIN or generate a fresh blob+PIN pair |
"no usable client config … run burrowee bootstrap" / "gateway public key required (--gw-pub)" | No gateway is paired yet (config.json names no gateway, or the gateway it names has no relay in its relays membership), or you passed --gw-pub explicitly pointing at a missing/unreadable file | Run burrowee bootstrap <blob> <pin> first — the matrix stores gateway key material inline in config.json now, so --gw-pub/--psk are only needed to override it explicitly, not to supply it |
| "daemon: config incomplete" | config.json is absent or was manually edited | Re-run bootstrap to regenerate it |
relays probe fails with "connect: no such file" | The transport daemon isn't running. probe is the one relay verb that asks the daemon for its answer — relays list, relays ping, gateways list and routes all read config.json directly (and ping dials the relay itself), so none of them needs a daemon | burrowee daemon or burrowee service install. Note relays use <id> still works — it persists to config.json first and only warns about the down daemon |
| "relay has no public address" / "cannot dial … over the LAN" | A relay with neither face — no public origin and no publishable LAN address — or a LAN address the CLI refuses to dial: a LAN dial is cert-pinned, so the relay needs a LAN cert fingerprint and the address must be wss:// | Nothing to set locally: there is no stored dial face to switch. The relay's operator has to publish the missing address (or its certificate). relays list marks the addresses in the second case (not dialable) |
| An edge with no public address seems unreachable after upgrading | Nothing is wrong, and nothing needs configuring — an edge the console minted with no domain has exactly one face, so an unpicked dial already uses its published, cert-pinned LAN addresses | Just connect/ssh as usual. The old relays via … lan step is gone because there is no longer a choice to record. Pass --relay wss://<address>@<short-id> only when you want one specific address for one connection |
| A relay you had pinned to LAN by hand now dials its public domain | The stored dial face is gone as of the v4 config. A relay that has a domain is dialled over that domain; the hand-set LAN pin did not survive the migration, and nothing warned at the time | Pass --relay wss://<address>@<short-id> per connection, or ask the relay's operator whether it should publish no public origin at all — see No stored dial face |
relays use <id> refuses: "is gateway …'s system relay, not one of its edges" | A default has to be an edge. On a host whose gateway has exactly one relay — every host still on a pre-relay-id config — that relay migrates in as the gateway's system path | Nothing is broken: the one relay it has is already what the gateway dials. There is no default to set until an edge is paired in with relays pair |
A relay that worked yesterday is gone from relays list, and --relay <that-id> no longer resolves | It was a gateway's second system relay. A gateway has one system path, so the extra one is no longer stored on it, and a relay no gateway names is collected on the next gateways resync | It returns as soon as the console makes it that gateway's system path or binds it as an edge, and the next resync brings it back. A local name you had set on that row (relays rename) does not survive the round trip |
| An app built on the CLI reports every relay as missing, or fails to read the matrix | routes --json is schema 2 and it is a clean break: the flat edges[] list and defaults.relay are gone, and gateways[].relays is now an object. Neither reader fails gracefully on that — one meets the array-to-object change as a JSON type error, the other silently degrades to an empty adjacency | Upgrade both CLIs together. burrowee-cli on its own, ahead of the on-top CLI, is exactly the state that produces this |
| After downgrading to an older CLI, every command fails — not just the relay ones | An older CLI cannot read a v4 config.json at all: it decodes strictly, so the per-gateway relays object is an unknown field and the whole file is rejected at startup | Restore the backup the migration left: cp ~/.burrowee/cli/config.json.v3.bak ~/.burrowee/cli/config.json. See Config homes & files |
| One relay stays broken even after reconnecting | A stuck cached carrier a reconnect won't recover | burrowee relays reset <id|host|name> retires every cached carrier for that relay across all gateways; the next use dials fresh |
Terminal floods with junk like ^[[<35;77;19M after a TUI crashed | A crashed full-screen program left the terminal's mouse/focus reporting switched on | Run any burrowee command on that terminal — startup clears leaked mouse and focus reporting on a real TTY (piped/--json output is untouched) |
| Need to re-pair from scratch | Stale or revoked pairing | rm -rf ~/.burrowee/cli, then bootstrap again with a new blob+PIN from the gateway operator (they can revoke the old pairing in their local console) |
Gateway
| Symptom | Cause | Fix |
|---|---|---|
Gateway shows awaiting approval in console.burrowee.com after pairing | A newly paired (or re-paired) gateway must be approved by the account owner before relays will serve it | Open the gateway's page in the console and click Approve on the banner — it starts serving within ~30 s |
Gateway doesn't appear online in console.burrowee.com | The relay URL in the blob isn't reachable from the gateway machine (outbound WSS) | Check /usr/local/burrowee/var/gateway/gateway.db exists (bootstrap wrote it), check the service logs for dial errors, and confirm outbound HTTPS reachability of the relay host — e.g. curl -v https://relay.example.com |
Local console unreachable (http://127.0.0.1:16518) | The daemon started with --console off, or isn't running at all | burrowee gateway service status; restart without --console off. burrowee gateway console-url prints the signed-in URL (admin-group membership required to read the token) |
| A session's lease reads unproven | The lease has never synced yet — that's a different fact from a lapsed lease | Usually transient; it proves itself on the next sync. If it persists, check the gateway's relay connectivity with burrowee gateway status |
service install fails on Linux: no systemd | Non-systemd init (OpenRC, runit, …) | Run burrowee-gateway foreground under your own supervisor (ExecStart=burrowee-gateway --no-open equivalent) |
bootstrap error: "wrong PIN" / decrypt failure | Blob and PIN must match exactly (case-sensitive blob, digit PIN) | Regenerate from console.burrowee.com → Gateways → Generate setup |
bootstrap refuses: an unmigrated legacy tree exists | A pre-0.2.0 ~/.burrowee/gateway tree was found and the gateway won't silently mint a fresh identity over it | burrowee gateway migrate --from <dir> (or re-run bootstrap --migrate-from <dir>) to adopt it; --accept-new-identity to deliberately start over |
| Need to re-bootstrap from scratch | Wiping config, relays, and identity | Stop the service first — macOS: sudo launchctl bootout system /Library/LaunchDaemons/com.burrowee.gateway.plist; Linux: sudo systemctl stop burrowee-gateway.service — then remove the config and data roots (/usr/local/burrowee/etc/gateway, /usr/local/burrowee/var/gateway) and bootstrap again (the console issues a new enrollment). Re-pairing an existing gateway is easier: the console's Pairing modal has a Re-pair tab that mints fresh material without the gateway dropping out of service |
Service logs — macOS: log show --predicate 'process == "burrowee-gateway"' --last 1h (plus /usr/local/burrowee/var/gateway/logs/); Linux: journalctl -u burrowee-gateway.service -f. Every stderr line is timestamped.
Edge
| Symptom | Cause | Fix |
|---|---|---|
| "awaiting approval" never flips to active | The operator approved a different edge, or is signed into the wrong account | Confirm the fingerprint shown in the portal matches the one bootstrap printed, and that the approver is the owner-tier account that minted the blob |
| Console unreachable | The edge only talks to the compiled-in Burrowee console — no override | Check the host's outbound HTTPS/WSS to the console; honor HTTPS_PROXY if behind a proxy |
doctor shows custom-domain cert ✗ for a while | DNS propagation + Let's Encrypt issuance take time | Confirm the _acme-challenge CNAME resolves to <slug>.acme.burrowee.net, then re-run doctor after a few minutes — the fault row echoes the real error and only points at --fix when --fix can actually repair it |
doctor exits 3 | One or more health rows show ✗ — the report tells you which; exit 3 is the "edge is red" verdict, distinct from exit 1 (the command itself failed) | Fix the named row, or doctor --fix for the repairs it can make (nginx front, host cert, a stopped daemon, a partially-adopted config) |
doctor rows read "cannot determine" / "not verifiable" | The unprivileged account can't read the root-owned config/data roots | Accept doctor's offer to re-read as root (it asks once, with consent), or re-run under sudo |
| Visitors to a custom domain see "not available yet" (HTTP 503) | The domain is routed but its tunnel is offline — this page (with Retry-After: 60) replaced the old redirect to the console, which looked like a login page | Bring the gateway/target back online; visitors' retries succeed as soon as the tunnel is up |
| Cloudflare 525s / TLS errors on the edge's own domain | Historically a drifted-empty host_fqdn config key skipped the host-cert check exactly where it mattered | Self-healing since 0.2.1: an emptied host_fqdn is restored from the host cert, and the cert check runs against the effective FQDN. If it persists, burrowee edge doctor — the host-cert row names the fault |
service install fails: systemd not available | Non-systemd init | Run burrowee edge run foreground or under a custom supervisor |
status: "enrolled; no config received yet" | The edge hasn't connected while approved | Run burrowee edge run while approved + connected, then re-check |
| Unhealthy nginx LAN front | Snippet missing, not applied, or nginx down | burrowee edge doctor --fix (install → apply → start), or re-run burrowee edge nginx reconcile --mode lan yourself — the nginx verbs self-elevate via sudo when needed |
| Need to re-pair from scratch | Wiping the edge identity | Stop the service, remove the config and data roots (/usr/local/burrowee/etc/edge, /usr/local/burrowee/var/edge), then bootstrap again — the console-side pending row must be re-minted |
Rotating the LAN cert invalidates pins
There is no rotate flag — to force a new LAN certificate, stop the service, delete /usr/local/burrowee/etc/edge/lan-cert/, and re-run sudo burrowee edge nginx reconcile --mode lan (see Rotating the LAN certificate). Every client/gateway that pinned the old fingerprint must re-paste a fresh blob afterwards (gateways heal automatically on the next endpoint report; CLI clients do not). It's a deliberate operator act, not part of routine renewal — the LAN cert is long-lived by design.