Backup Jobs (Job Queue)
Every backup, restore, export, and related task is a job stored in the dashboard database. Agents claim work by polling (site-scoped). The Jobs page is the live operator view.
Job lifecycle
| Status | Meaning |
|---|---|
| Pending / Queued | Created; waiting for an agent to claim it |
| Running | Agent executing; may show file counts, restic progress, elapsed time |
| Completed | Finished successfully; remains until cleared or cleanup policy |
| Failed | Error from agent or system — open details and agent logs |
Where jobs come from
- Dashboard Run / Backup all (primary repository per site)
- Schedules (minute runner)
- Restore / Export / archive restore
- Public API, the built-in MCP server, or a connected chat app
The Restore page also shows a focused Active job card for restore/export types; full history always lives under Jobs.
Filters and search
Default filter is Recent (about 7 days + active work). Switch to Active only, All statuses, or a single status. Search matches type, site, agent, progress text, or job id. Stat tiles jump to common filters.
Actions
- Refresh — pull now (the page also live-updates).
- Clear finished — remove completed/failed rows from the list (does not delete backup data).
Admins can enable automatic retention under Settings → Jobs cleanup (delete finished jobs older than N days).
Restic under the hood
Reaver uses restic on the agent for:
- Incremental, deduplicated snapshots
- Optional client-side encryption of repository data (off by default)
- Restore by snapshot id
Local and remote restic passwords are both dashboard-managed; see Security and Agents.
Destination timeouts and retries
Remote (Offsite) backups and restores do not hang forever when an SSH/S3 host is unreachable (for example mid-reboot):
- SSH uses a short ConnectTimeout (default 30s).
- Probe/connect restic commands have a wall-clock limit (default 60s).
- Connect-like failures are retried up to 2 more times (3 attempts total), with short backoff (5s then 15s).
- After retries fail, the job is marked Failed with a clear message (destination may be rebooting, offline, firewalled, or credentials wrong). Failed-backup email uses that message.
- Wrong password / SSH auth failures are not retried as connect errors.
Progress text may show “Retrying destination connection (attempt 2/3)…” while retries run.
Tune timeouts on the agent host via environment variables documented under Agents → Destination connect timeouts.
| Variable | Default | Role |
|---|---|---|
REAVER_SSH_CONNECT_TIMEOUT |
30 |
OpenSSH connect timeout (SFTP) |
REAVER_DEST_CONNECT_TIMEOUT |
60 |
Probe / short-command process timeout |
REAVER_DEST_CONNECT_RETRIES |
2 (extra) |
Extra connect retries after first attempt |
REAVER_DEST_BACKUP_TIMEOUT |
unset | Optional hard cap; unset means run as long as restic keeps producing output |
Troubleshooting failed or stuck jobs
- Confirm the agent for that site is online.
- Check destination credentials (Offsite → Test) for remote backups.
- Confirm a primary repository is set if Dashboard Run does nothing useful.
- If the job sits on “Connecting to remote destination…”, check Offsite credentials and host reachability, then re-queue.
- Read the failed job message: connect-timeout text means fix host reachability or Offsite settings, then re-queue.
- Enable Debug logging under Settings for
data/debug.log. - On the agent host, inspect worker logs /
journalctlfor the reaver-agent service.