PBS 4.2 Parallel Sync Jobs Cut Offsite Backup Time

PBS 4.2 parallel sync jobs let multiple replication targets run concurrently, cutting nightly backup windows in half. Learn to configure and tune them correctly.

Proxmox Pulse Proxmox Pulse
9 min read
Parallel streams of glowing data flowing simultaneously from server racks to offsite destinations.

Proxmox Backup Server 4.2 shipped parallel sync job controls in early 2026, and if you run multiple offsite targets — a colocation box, a friend's homelab, an S3-compatible bucket — you can cut your total nightly replication window from three hours to under one. Here's exactly how to configure it and where it will bite you if you're not careful.

Key Takeaways

  • Parallel jobs: PBS 4.2 lets multiple sync jobs run simultaneously instead of queuing sequentially.
  • Per-job workers: Each sync job gets its own max-workers setting, controlling how many parallel chunk transfers run internally.
  • WAN impact: Two parallel jobs with 4 workers each will fully saturate a 100 Mbps uplink — rate-limit them.
  • RAM cost: Each additional worker adds roughly 50–100 MB of working memory on the source PBS node.
  • Scheduling tip: Even with parallel support, stagger start times by 15 minutes so disk I/O doesn't spike simultaneously.

What Changed Between PBS 4.1 and 4.2

Before PBS 4.2, the sync subsystem used a single global job queue. When Job A was syncing 500 GB to a remote PBS node, Job B for a different target sat idle in the queue waiting for A to finish. On a server with four datastores and four remotes, that meant three of them were always blocked.

PBS 4.2 decouples sync jobs from the global queue. Each job now runs in its own async task context, and the web UI and API both expose a max-workers parameter per job. This isn't just a UI toggle — the underlying sync protocol was updated to pipeline chunk requests, so jobs with max-workers greater than 1 actually upload and download multiple chunks in flight simultaneously.

The practical result: running two sync jobs to different remotes at the same time now works without either one stalling. On a homelab server with a 500 Mbps symmetric fiber uplink, two parallel 4-worker jobs complete in roughly the same wall-clock time as one sequential job used to take.

How Parallel Sync Workers Actually Work

There are two levels of parallelism here, and confusing them leads to over-provisioning.

Job-level parallelism is about running multiple sync jobs at the same time. PBS 4.1 would serialize these; PBS 4.2 does not.

Worker-level parallelism is about how many parallel chunk transfers happen within a single sync job. This is the max-workers setting. The default is 1 (sequential chunk fetch), but you can set it as high as 16. In practice, values above 4 offer diminishing returns unless you have a high-latency, high-bandwidth WAN link where pipelining helps amortize round-trip time.

The interaction matters: if you run three sync jobs in parallel, each with max-workers 4, you have up to 12 concurrent chunk operations competing for your uplink. Plan accordingly.

How to Set Up Parallel Sync Jobs in PBS 4.2

Verify Your PBS Version

dpkg -l proxmox-backup-server | grep ^ii

You need proxmox-backup-server 4.2.x or later. If you're on 4.1 or earlier, upgrade first:

apt update && apt dist-upgrade

Reboot after the upgrade. PBS 4.2 ships a new proxmox-backup-proxy binary, and the old one stays running until the next restart.

Add Your Remote Targets

If you haven't configured remotes yet, you need one per offsite destination. In the PBS web UI: Remotes → Add. Via CLI:

proxmox-backup-manager remote create offsite-pbs \
  --host 203.0.113.42 \
  --auth-id sync-user@pbs \
  --password 'your-token-secret' \
  --fingerprint AA:BB:CC:DD:EE:FF:00:11:22:33:44:55:66:77:88:99:AA:BB:CC:DD:EE:FF:00:11:22:33:44:55:66:77:88:99

Replace 203.0.113.42 with your remote PBS host IP, and pull the TLS fingerprint from the remote PBS web UI under Administration → Certificates. For an introduction to the overall PBS workflow before adding parallelism, see our guide to automated backups with Proxmox Backup Server.

Configure Sync Jobs with max-workers

Navigate to Datastore → [your-datastore] → Sync Jobs → Add. The Max Workers field is new in 4.2 and defaults to 1.

Via CLI, creating a sync job with parallel workers:

proxmox-backup-manager sync-job create daily-offsite \
  --store local-backup \
  --remote offsite-pbs \
  --remote-store remote-backup \
  --schedule 'daily' \
  --max-workers 4 \
  --remove-vanished false

To update an existing sync job in place:

proxmox-backup-manager sync-job update daily-offsite --max-workers 4

Verify the change took effect:

proxmox-backup-manager sync-job list --output-format json | jq '.[] | {id, "max-workers"}'

Run a Test Sync and Watch the Bandwidth Graph

Trigger a manual run before depending on the schedule:

proxmox-backup-manager sync-job run daily-offsite

In the PBS web UI, open Dashboard → Task History and watch the sync task. A 4-worker job syncing a 200 GB datastore over a 500 Mbps uplink should complete in roughly 55–60 minutes, compared to 3.5–4 hours for a 1-worker job on the same link.

Tuning max-workers for Your Network

The right max-workers value depends on three variables: link latency, available bandwidth, and how many jobs run simultaneously.

Scenario Recommended max-workers Reason
LAN sync to local PBS replica 2 Low latency, high bandwidth — extra workers add little
100 Mbps fiber, 1 sync job 3–4 Saturates the link efficiently
500 Mbps+ fiber, 1 sync job 4–6 Pipelining compensates for chunk RTT
500 Mbps fiber, 2+ parallel jobs 2–3 per job Avoid oversubscribing your uplink
High-latency WAN (>80ms RTT) 6–8 Parallelism amortizes round-trip overhead
S3-compatible target 4 S3 API concurrency limits vary by provider

Run iotop on the PBS host and watch your router's WAN graphs during a test sync. If the uplink hits 100% and you see retransmits, reduce workers. If it sits at 40%, add workers.

Running parallel jobs without rate limits will saturate your WAN during peak hours. PBS 4.2 lets you set a bandwidth cap per sync job:

proxmox-backup-manager sync-job update daily-offsite \
  --bandwidth-limit 50000

That value is in KB/s — 50,000 KB/s is roughly 400 Mbps. For overnight jobs where you want full throughput, remove the cap:

proxmox-backup-manager sync-job update daily-offsite \
  --schedule '02:00' \
  --bandwidth-limit 0

Setting --bandwidth-limit 0 disables the limit entirely. Use non-zero values for daytime maintenance syncs when you need to share the link with production traffic.

Common Pitfalls With Parallel Sync Jobs

Spinning disk I/O thrashing. If your backup datastore lives on HDDs, 8+ workers will thrash the seek heads. Chunk reads become random I/O and throughput drops below single-worker performance. Keep max-workers at 2–3 for HDD-backed datastores regardless of WAN speed — the bottleneck is the disk, not the link.

RAM exhaustion on the source node. Each worker holds chunks in memory during transfer. A 4-worker sync job uses approximately 400 MB of RSS on the source node. Three parallel jobs at 4 workers each means roughly 1.2 GB of extra overhead on top of normal PBS operation. Run free -h before raising worker counts on memory-constrained systems.

Prune vs. sync timing conflicts. If a prune job fires while two sync jobs are active, PBS will lock the affected datastore chunks. The sync jobs don't fail outright, but they retry locked chunks repeatedly and throughput collapses. Schedule prune jobs at least 30 minutes after your sync window ends — not overlapping.

Accidental snapshot deletion with remove-vanished. With parallel workers, the delete pass at the end of a sync completes faster. If you have --remove-vanished true set with an incorrect namespace filter, it will wipe snapshots on the remote end faster than you expect. Leave it false until you've confirmed the filter on a dry run, then flip it deliberately.

For the foundational PBS configuration before layering parallelism on top — datastore creation, retention schedules, and namespace design — see the automated backups with Proxmox Backup Server guide.

How to Monitor Parallel Sync Job Health

PBS exposes per-job task history and throughput stats via the management CLI:

proxmox-backup-manager task list --typefilter syncjob \
  --output-format json | jq '.[] | {upid, status, "worker-id"}'

For failure alerting, configure PBS email notifications so a broken sync doesn't go unnoticed for days:

proxmox-backup-manager notification-matcher create sync-alerts \
  --mode 'always' \
  --target 'email-admin'

This sends email on both success and failure — noisy but useful when first tuning parallel jobs. After a week of clean runs, switch --mode to error to quiet it down.

The proxmox-backup-server Prometheus endpoint at /metrics exposes pbs_sync_duration_seconds and pbs_sync_transferred_bytes per sync job. If you already have a Prometheus scrape setup in your homelab, adding PBS as a target takes under five minutes and gives you duration trend graphs that will catch a degraded offsite link before it becomes a missed backup.

Securing Parallel Sync Connections

Parallel sync jobs open more concurrent TCP connections to your remote PBS node. Lock port 8007/tcp on the remote firewall to your source IP only — not to the world. If you're tunneling over WireGuard between sites, set the tunnel MTU to 1380 or lower to avoid packet fragmentation with multiple simultaneous flows.

For a full review of Proxmox firewall rules and SSH hardening that applies equally to PBS nodes, the Proxmox VE hardening guide covers iptables chains, fail2ban, and certificate-based authentication.

Conclusion

PBS 4.2 parallel sync is a genuine operational upgrade: set 3–4 workers per job, run two offsite targets concurrently, and your nightly backup window drops from hours to under one on most homelab uplinks. Set per-job bandwidth limits, watch RAM usage on the source node, and keep prune schedules from overlapping with sync windows. Once parallel sync is running cleanly, the next move is wiring the PBS Prometheus endpoint into Grafana so you can catch duration regressions before a backup target falls silently behind.

Share
Proxmox Pulse

Written by

Proxmox Pulse

Sysadmin-driven guides for getting the most out of Proxmox VE in production and homelab environments.

Related Articles

View all →