Data streaming configuration for catalog
Dynamic streaming refers to a set of backup streams that run in parallel and dynamically distribute image database subdirectories among themselves during the backup operation. This approach optimizes and accelerates the backup of large catalog data sets.
The Allow multiple data streams option is enabled by default for new catalog policies, and the user must configure the Maximum number of streams value.
Maximum number of streams- This parameter defines the number of backup streams used to back up NetBackup catalog data. It can be configured when creating a catalog policy from the Policy Attributes tab in the NetBackup web UI.
Default value: 4 streams
Supported range: 1 to 64 streams
For catalog policies created prior to NetBackup 11.2, the Allow multiple data streams option is disabled by default and must be explicitly enabled by the user. After enabling this option for an existing policy following an upgrade, a full backup is required to avoid backup job failures.
To enable the Allow multiple data streams and configure the Maximum number of streams, use the NBU-Catalog policy configuration UI or the bpplinfo command.
With catalog dynamic multi‑streaming feature enabled, configuration files are backed up under stream #1 and image .f files are backed up from stream #2 through stream #N.
For catalog policies that use dynamic streaming, multiple backup images are created for a single catalog backup. The number of images that are created corresponds to the the value that is set for the Maximum number of streams option.
Because a single catalog backup consists of multiple images, NetBackup groups these images together. Any operation performed on one image in the group is automatically applied to all other images associated with the same catalog backup. For example, if the maximum number of streams is set to four and one image from a catalog backup is expired, the remaining three images are also expired.
Image grouping applies to the following operations:
Image expiration
Image import
Image duplication
Image verification
Set primary copy
Note:
Image grouping does not apply to the Browse and Restore operations and to the following commands: bpimage, bprestore, bpcatlist, bpcatarc, bpcatrm, cat_export and cat_import. In these cases, operations must be performed on catalog backup that comprises multiple sibling images, and can be uniquely identified by image_group_key. The bpimagelist command can be used to determine the group ID for images of the same type.
Refer to the bpimagelist command documentation.
If the Allow multiple data streams option is enabled or the Maximum number of streams option is modified, a full backup is required.
If a full backup is not performed, jobs associated with the policy may fail. To avoid job failures, run a manual full backup immediately after saving the policy changes.
For catalog policies created prior to NetBackup 11.2, the Allow multiple data streams option is disabled by default and must be explicitly enabled by the user.
After enabling this option for an existing policy following an upgrade, a full backup is required to prevent backup job failures.
Updating the Maximum jobs per client option
- On the web UI, go to Hosts > Host Properties.
- Select the primary server and click Edit primary server.
- Go to Global attributes.
- Update the Maximum jobs per client option to a value greater than or equal to the highest value that is set for the Maximum number of streams option across all the catalog policies that are configured on that primary server.
Note that this default value is set to 1 which needs to be updated so that dynamic multi stream catalog backup jobs are successfully completed.