We've been running Commvault 11 for a few years and recently got an Aux Copy to AWS S3 working and tested so we can recover in a full DR scenario. Now I'm trying to figure out the best settings to make sure we're getting an efficient data transfer and storage. I've read through many articles and forum posts but nothing that ties everything together and am hoping posting here will help organize what I've read.
Our environment is two sites with one media agent local to each that handles backups for VMs, file agents and DB agents. Every day the previous night's backups are sync copies to the other site. We use dedupe locally and Aux copies are DASH. A single CommServe controls everything and all data is AES encrypted.
For the copy to AWS S3, all data is sent from media agent A. It sends jobs from the Primary copy for storage policies that use it's deduped disk library, and uses the Secondary copies for jobs that come in from the remote site media agent B. That works well over our 1Gb synchronous internet connection.
What I'm struggling to iron out are the detailed settings like block sizes and how to get the most efficient dedupe and job thoughput.
The different block level dedupe factors we have set are:
Local global dedupe DBs: 128KB
AWS S3 global dedupe DB (on media agent A): 512KB
Storage policies: 128KB
The block sizes on the copies' data paths are:
Primary and Secondary copies to disk libaries (on site): Use media type settings (64KB)
AWS S3 copy: 1024 (automatically set when creating the copy)
Based on the last bullet point in link 1 below, should I change the storage policies to have a block level of 512KB or is the default 128KB like link 2 shows OK?
Similarly, should the data path block size for the S3 cloud lirbary be set to 512KB instead of 1024? I only found reference to the 1024 in link 3 for tuing 10Gb+ connections.
My plan was to leave storage policy block sizes at 128KB and set the cloud library data path to 512KB. Any advise is welcome. Thanks!
Note: The S3 cloud copies are DASH Network Optimized and Space optomized Aux Copy is checked.
Link 1: http://documentation.commvault.com/commvault/v11/article?p=93056.htm
Link 2: http://documentation.commvault.com/commvault/v11/article?p=12411.htm#o12418
Link 3: http://documentation.commvault.com/commvault/v11/article?p=8629.htm