# BTX node guardian

A small script that keeps a BTX node from getting quietly stuck, and tells you the
truth about why it is still when it is still.

Free to copy, change and redistribute. Written by the easyBTX team from a week of
live incidents on `api.btxscan.io`; every rule in it is something the network taught
us the hard way.

## The problem it solves

A trusted-mirror node that stops advancing looks identical in two completely
different situations:

| What you see | What is actually happening | What helps |
|---|---|---|
| Tip still, `matmul trusted mirror stall` in the log every minute, header above your tip | **You are AT the signed frontier.** The attestor has signed nothing newer. The whole network is waiting. | Nothing. Wait. |
| Tip still, same log line, same header | **Signed work exists that you have not collected.** Your node is behind and not catching up. | Dial archive peers. |

The difference is one number: `getmatmulattestedtip → signed_frontier.blocks_behind`.
`0` means the first case, anything higher means the second.

This matters more than it sounds. On 2026-08-19 the network's attestor was offline for
about 100 minutes while GPU consensus nodes ran 43 blocks ahead on unattested work.
Every watchdog without this distinction reported a stall for the whole period and
dialled peers that had nothing to give — and an operator who is warned wrongly often
enough stops reading the warnings.

## What it does

Every run it reads your node and decides one of:

- `ok` — nothing to say.
- `network_frontier_quiet` — the signed frontier itself has not moved for an hour.
  Not your node's problem; reported, never acted on.
- `no_qualifying_peer` / `attestation_missing` / `body_missing` — you are behind the
  frontier. It dials the archive peers, at most once every 10 minutes.
- `electrs_lagging`, `btxd_down`, `rpc_unreachable`, disk pressure — reported.

It writes one JSON file (`health.json`) with the whole picture: blocks, headers,
signed frontier, lag, quorum, peer and archive census, who feeds you, who you serve,
how long the tip and the frontier have each been still.

## What it will never do

- **Restart or stop your node.** A restart discards the peer set that is usually your
  only attestation source, and an unclean stop has bricked a snapshot datadir. Crash
  recovery belongs to `Restart=` in your service file — a different thing from
  reacting to a stall.
- **Touch your datadir, conf, or signer keys.** Changing `matmultrustedpubkey` over an
  existing chain makes btxd refuse to start, and the `-reindex-chainstate` it suggests
  deletes the snapshot chainstate *before* failing the same check. A pruned
  snapshot node cannot full-reindex out of that.
- **Hammer peers.** A signer serves only the last 16 blocks and bans you for 24 hours
  after 32 ignored requests. Historical scans belong on archives.

## Install

One line, and it finds your node by itself:

```bash
curl -fsSL https://easybtx.com/install-node-guardian.sh | bash
```

Or by hand, if you would rather read everything first:

```bash
curl -O https://easybtx.com/btx-node-guardian.sh
chmod +x btx-node-guardian.sh
BTX_CLI=/path/to/btx-cli BTX_DATADIR=$HOME/.btx ./btx-node-guardian.sh
cat ~/.btx-guardian/health.json
```

Then run it every 5 minutes. With cron:

```bash
*/5 * * * * BTX_CLI=/path/to/btx-cli BTX_DATADIR=$HOME/.btx /path/to/btx-node-guardian.sh
```

Or with systemd, which also survives reboot:

```ini
# /etc/systemd/system/btx-guardian.service
[Unit]
Description=BTX node guardian
After=btxd.service

[Service]
Type=oneshot
Environment=BTX_CLI=/path/to/btx-cli
Environment=BTX_DATADIR=/path/to/datadir
ExecStart=/path/to/btx-node-guardian.sh
```

```ini
# /etc/systemd/system/btx-guardian.timer
[Unit]
Description=Run the BTX node guardian every 5 minutes

[Timer]
OnBootSec=3min
OnUnitActiveSec=5min
Persistent=true

[Install]
WantedBy=timers.target
```

```bash
sudo systemctl enable --now btx-guardian.timer
```

## Configuration

All optional, all environment variables:

| Variable | Default | What |
|---|---|---|
| `BTX_CLI` | `btx-cli` | path to your btx-cli |
| `BTX_DATADIR` | `~/.btx` | node datadir |
| `GUARDIAN_STATE` | `~/.btx-guardian` | where it keeps counters between runs |
| `GUARDIAN_BEACON` | `~/.btx-guardian/health.json` | the JSON it writes |
| `BTX_ARCHIVES` | the published set | archive peers to dial, space-separated |

## Two things worth knowing about your peers

A trusted mirror only asks peers that are **manual (`addnode`) or `noban`** for
attestations and block downloads. A peer that is merely connected is never asked for
anything. If your node is starving with a healthy-looking peer list, this is usually
why:

```
addnode=<archive>:19335
whitelist=in,out,noban@<archive-ip>
```

The `in,out` matters: a bare `whitelist` applies to incoming connections only, and the
connection `addnode` creates is outgoing.

## Requirements

`bash`, `python3` (for JSON parsing), and a btx-cli that can reach your node. No
packages to install.