The client direct backup fails with status code 50 due to client memory exhaustion
When running a backup job, you may observe a failure with status code 50: client process aborted in the Activity Monitor. This issue is often caused by insufficient memory on the client system, leading to the termination of the nbostpxy process by the operating system.
Check for the following on the Activity Monitor:
Info nbjm: started backup job for client <client_name>, policy <policy_name>, schedule Full on storage unit <storage_unit> Info bpbrm: started process Info bpbrm: connecting Info bpbrm: connected Info bpbrm: begin writing Critical bpbrm: unexpected termination of client <backup_id> Error bptm: media manager terminated by parent process Info dbclient: done. status: 50: client process aborted
Check /var/log/messages for out of memory events. You may see entries like:
Out of memory: Killed process <PID> (nbostpxy) total-vm:<value>kB, anon-rss:<value>kB, ...
Run free -g on the client. If the output shows zero free memory, it confirms that the system is under memory pressure:
total used free shared buff/cache available Mem: 15 12 0 0 2 1 Swap: 7 6 1
Cause:
The nbostpxy process allocates a large write buffer (default: 512 MB per instance). On systems with limited memory, this allocation can lead to excessive memory usage and OOM conditions, causing intermittent backup failures.
Resolution:
To resolve this issue, reduce the size of the write buffer that is allocated by nbostpxy. By default, each nbostpxy instance allocates 512 MB for writing. You can lower this value to prevent excessive thrashing by setting the maxWriteBufferSizeMB and maxPendingWriteDataSizeMB parameters in the spanfs.conf file.
For example:
{
"logLevel": 5,
"logSize": 30,
"logPath": "/var/log/spanfs",
"writeParams": {
"maxWriteBufferSizeMB": 16,
"maxPendingWriteDataSizeMB": 8
}