Number of data streams

Last post 01-13-2015, 6:10 AM by ignesi. 8 replies.
Sort Posts: Previous Next
  • Number of data streams
    Posted: 03-07-2012, 11:22 AM

    How do you determine how many data streams you have availabe to you? I have 2 media servers both are fiber channel connected to my SAN with 4 Fiber ports each. My understanding is that each port is dual streamed (?) thus giving me 16 maximum data streams at any one time. I cannot find anything that would confirm or deny this and could really use some help trying to determine how many actual streams I have availabe.

  • Re: Number of data streams
    Posted: 03-07-2012, 2:44 PM

    Start with stream 2x the number of spindles in the mountpath(s)

  • Re: Number of data streams
    Posted: 03-07-2012, 3:04 PM

    From the sending side, or the receiving side?

    What do you mean by spindles? if from the receiving side, these are luns that are multiple disks used to create that lun.

  • Re: Number of data streams
    Posted: 03-07-2012, 3:12 PM

    So if the lun has 5 disks i would set the streams to 10 at first.  From there you can run some performance testing to see if going higher nets any different results.  You could also test 5 to see if any performance difference is experienced. 

  • Re: Number of data streams
    Posted: 03-07-2012, 3:43 PM

    I will test those settings and see if it improves it any. The issue I am having is I have 2 VSA (VM) sub-clients, a full backup on one takes 16-18 hours, the other over 24 hours to complete. I am looking at readers vs streams to see if I can improve this any.

    Thank you for your help so far.

  • Re: Number of data streams
    Posted: 03-07-2012, 3:49 PM

    each reader is a VM.  So if 10 readers are set and there are 10 streams, 10 VMs get backed up in parallel.

  • Re: Number of data streams
    Posted: 09-08-2014, 3:24 PM

    To revisit this from long ago.

    I have 3 media servers that can backup my VM's. I back up about 12tb nightly in total. I backup straight to a data domain appliance on roughly a 20G pipe.

    I am trying to determine how I maximize my throughput, I get a lot of pending resources thru the night. I had the data streams per media server set to 100 max, now to 200 max, on the storage policy I used to have it set to 10 streams but after reading some other threads I am told I could go as high as 50 streams (?). I set it to 25 to test.

    Each sub-client in that policy is set to 5 streams (5 simultaneous backups).


    Am I headed in the right direction? How far can I boost this without compromising the rest of my backups (Non VM) like databases and no VM clients?

  • Re: Number of data streams
    Posted: 09-09-2014, 3:50 AM

    1st of all the resticting number of threads / streams /readers (which is mor or less the same) is set in the StoragePolicy you assign to a subClient.

    If you assign multiple subClients to one StoragePolicy (which is pretty normal), the number of simultaneous backupstreams can sum up to the number of streams you defined within the StoragePolicy.

    if you use multipe StoragePolicies at one time for backup (simultaneously started subClients to different SPs) the overall parallel backupstreams can sum up to the sum of streams defined in the used SPs.

    If you use only one MediaAgent this might restrict the number of parallel streams to the number defined for the MediaAgent, even if there are more streams defined/available within the SPs you use.
    If using multiple MAs for the Backup, each MA can handle the number of steams defined for it.
    So the number of simultaneous backupstreams might be larger than the number of streams per MA.

    There is another counter for the number of jobs, that can be handled in parallel by the commserve, which might also affect the number of streams in your environment.

    last you find the number of reader in the subclient properies. In case of VSA, each VM can only use one reader (no further split possible). Using multiple reader will try to start more VMs for Backup in parallel (up to the number defined and streams available in SP and acting MA).

    Using multiple VSA for backing up a subclient might increase the throughput. If virtual VSA are used, more ESX Server can handle the load.

    Whether your environment can handle the load of 50 or 100 or 200 parallel VM backups is most of the time not a question of bandwidth but a question of the number of random / sequential I/Os which can be handled simultaneuosly by your source and destination storage.

    The FC bandwitdh available might not be the problem.

    When testing or doing backups try to shorten the backuptime for an individual VM, since a VM SnapShot has to be presend during backuptime and the longer it takes, the longer it'll take to remove it once the VM backup is done

    try to split VMs to subClients according to their size, so the VMs in a subclient are comparable. This might result in "backup done" at mostly the same time for all VMs in a subClient. Otherwise interrupts and errors in your Backupenvironment will have a strong impact on the backup and the portion to be redone. A subClient can only be backed up once at a time. so if you have VMs which take 24h+, all VMs in the same subclient cannot be backed up once a day since the subClient's still active.



  • Re: Number of data streams
    Posted: 01-13-2015, 6:10 AM

    To what model Data Domain are you backing up to?

    Keep in mind that the stream count limits vary per model of Data Domain.

The content of the forums, threads and posts reflects the thoughts and opinions of each author, and does not represent the thoughts, opinions, plans or strategies of Commvault Systems, Inc. ("Commvault") and Commvault undertakes no obligation to update, correct or modify any statements made in this forum. Any and all third party links, statements, comments, or feedback posted to, or otherwise provided by this forum, thread or post are not affiliated with, nor endorsed by, Commvault.
Commvault, Commvault and logo, the “CV” logo, Commvault Systems, Solving Forward, SIM, Singular Information Management, Simpana, Commvault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, SnapProtect, ROMS, and CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.
Copyright © 2018 Commvault | All Rights Reserved. | Legal | Privacy Policy