Performance tuning of catalog backup with respect to dynamic multi-streaming
Optimizing catalog backup performance requires careful tuning of stream counts and resource allocation to balance speed and system load. This topic details key considerations for configuring catalog dynamic multi-streaming on the primary server. It helps you choose an appropriate number of catalog backup streams for your NetBackup environment.
Catalog dynamic multi-streaming is a read-dominated workload that runs on the primary server, with the primary itself acting as the client being backed up.
During a catalog backup, the primary server crawls the catalog volume, streams data from disk through the page cache, and forwards it to the configured storage target. The operating system must therefore provide enough CPU, memory, and storage bandwidth so that none of the three becomes a bottleneck while normal NetBackup daemons and OS services continue to respond.
For each stream that you configure, NetBackup starts one backup worker on the primary server that reads through a portion of the catalog data and sends it to the configured storage target. The primary server continues to perform all of its normal duties at the same time, scheduling jobs, managing running backups, allocating drives and network resources, answering queries from the catalog database, and serving the administration UI and API.
The host must therefore have enough CPU, memory, and disk headroom to handle both the catalog backup and its day-to-day workload without slowing down.
Use the following rules to choose a stream count and to know when to stop adding more. This can vary across different domains based on hardware, load, and so on.
Start small and increase gradually
Begin with the default of 4 streams. Gradually test with increasing stream count and stop once the next step delivers less than ~5 % additional benefit.
Catalog backup runs on the primary server and competes with normal NetBackup operations, so the goal should be to complete the operation within the backup window and not as fast as possible.
Cap streams to a fraction of available cores
As a starting point on the primary server, allow no more than 1 stream per 2-4 cores. Always reserve at least 4 cores for NetBackup services and 2 cores for the OS, even on small systems.
Plan memory per stream
Allow roughly 200-400 MB of additional resident memory per stream on top of the steady-state footprint of the NetBackup daemons, the catalog database, and the operating system.
Plan catalog image disk
Catalog multi stream backup requires extensive read of image database. For catalog image database "./netbackup/db/images/", use SSD or other high-end storage on the primary server. With NFS storage, performance will be low.
Plan media storage
Catalog backup throughput is ultimately bounded by how quickly the storage target can absorb the data that is being written. After the primary server is well sized for CPU, memory, and image database storage, the storage target's write speed may become a bottleneck.
Provision SSD-backed disk pools, NVMe-class arrays, or other high-throughput storage for the catalog backup target, especially on multi-TB catalogs.
AdvancedDisk STU consideration
Set the stream count greater than or equal to the number of volumes in the disk pool. If required enable the ActiveWriteStream attribute on AdvancedDisk pool so that write is round-robined across disk pool volumes for parallel IOPS and better performance:
nbdevconfig -changedp -stype AdvancedDisk -dp dp_name -setattribute ActiveWriteStream
Tape STU consideration
NetBackup does not currently support tape multiplexing for catalog backup. Keep the stream count same as the number of drives in the tape library or set it to a lower number.