Auxcopy To Tape Speed - Hardware Compression?

Last post 06-26-2017, 9:22 AM by Totopolis. 7 replies.
Sort Posts: Previous Next
  • Auxcopy To Tape Speed - Hardware Compression?
    Posted: 03-26-2012, 4:03 PM

    I'm still struggling with getting good throughput to tape of large Synthetic Full backups in spite of seeing fantastic results on verify operations on the same backups.

    I have the lookahead reader,nNumPipelineBuffers, and dwMaxAsyncIoRequests keys set, I have unbuffered IO set on the mount paths, the mount paths aren't particularly fragmented and I have a lot of spindles all configured as per the building blocks paper.

    As I mentioned above if I run a verify on the jobs I see fantastic consistent speeds, it's only when doing an aux copy that the speeds seem to float all over the place.

    I noticed that when doing aux copies from dedupe primary, my tapes never hold more than their native capacity, so LTO4 media will always be marked as full at around 790mb, LTO5 at around 1400-1500mb.

    Something I just spotted in another thread was a suggestion to disable hardware compression on the data path to the tape library on the aux copy, if using software compression.

    Has anyone tried this?

    Also I'm a little confused if I actually am using software compression or not?

    If I look at a running job it say "OFF", if I look on a subclient it says "Use Storage Policy Setting" and if I look on my Global Dedupe polict, which all my Storage Policies dedupe against, it says Software Compression is "On" - which is it?! Tongue out

    Oh just to add, using 256kb block & 16gb chunk on the aux copies.

    MA is 2008 R2 and everything is Commvault 9.0 SP5a.

  • Re: Auxcopy To Tape Speed - Hardware Compression?
    Posted: 03-26-2012, 4:19 PM

    When you say large, do you mean it holds a lot of files or just that its size is big? Or both? I'm asking because I'm wondering if it could be something related to my issue which I posted here:

    Even though my aux copy is from disk to disk, you can clearly see that something in the Simpana engine seem to be slowing things down when there's a huge amount of files. I'm in contact with our technical rep to try and figure things out. He thinks it might be related to the index which has to be updated a lot more when there are many files. I'll let you know if I get any update on this.

    Meanwhile, have you tried running a similar job in size but with only a few big files in it to see if you get the same kind of speed? Hopefully you have another synthetic that can qualify. I used a synthetic of Oracle flat files in my test...


  • Re: Auxcopy To Tape Speed - Hardware Compression?
    Posted: 03-26-2012, 4:41 PM

    In our case around 7tb and 8m or files, but keep in mind this is re-hydrating from disk to tape so the DASH issues that I think your thread is hinting at shouldn't come into it should they?

    AFAIK in our case the auxcopy job has no visibility of the content of the job, it just moves blocks/chunks/index from one copy (disk) to the other (tape) - doesn't it?

    All I know for sure is that if I run a veryify on a Synthetic Full job on the primary disk copy it'll run at, say, 300-400MB/Sec consistently, yet doing the aux to tape some parts can be a fairly stable 120-140MB/Sec, others can be 40MB/Sec and I often see quite lenghy pauses between blips of activity.

    In either case, as the source primary job is a Synthetic Full I belive I'm only dealing with a single stream so streams shouldn't be the issue.

  • Re: Auxcopy To Tape Speed - Hardware Compression?
    Posted: 03-26-2012, 11:29 PM

    You're right, I'm using DASH! Read your post quickly while at the job and didn't have much time to think it through!

    I'm not an expert at transfer speed to tape but if it behaves somewhat the same as FTP, file copies, etc. then when it copies 1 big file, it will go much faster than when copying a lot of smaller files since it doesn't have to write as much metadata.

    I'm meeting with my tech rep tomorrow. Hopefully he has updates on my speed issue... I'll try to think to ask him about yours and see if it's somewhat related.


  • Re: Auxcopy To Tape Speed - Hardware Compression?
    Posted: 03-27-2012, 2:18 PM

    What kind of tape drive are you using stand alone or a tape libary? Are you using any type of encryption?  Are you using GridStore how is your disk set up. Is your data on one media agent and another media agent is doing the AUX copy. If it is the data is going over just like it would via UNC path.

    Somet things that helped us out

    Enabled combine source data streams and turn on mutilplexing

    We have everything going to disk in 1 or 2 tb chuck sizes. Then for aux copy to tape we double it. That seemed to improve the shoeshining of the tape devices.

    I heard from Commvault not to check the hardware compression as the tape device is set up that way already. I think you have to go in and disabled it as its a feature.

    We have are using Commvaults encyrption (AES) and see speeds around 200GB/hr to tape.  I am sure we woudl get faster speeds without encryption.

  • Re: Auxcopy To Tape Speed - Hardware Compression?
    Posted: 04-17-2012, 2:26 AM

    Set tape block size (LTO-4 and above) to at least 512KB.

    Disable the tape Hardware compressions as by DEFAULT commvault will ignore the software compression IF to tape and h/w compression is enabled.


  • Re: Auxcopy To Tape Speed - Hardware Compression?
    Posted: 06-20-2013, 12:48 PM

    I have pretty much tweaked my aux copy to tape speeds to as good as I think they are going to get (and it is quite nice), but the question I am not seeing an answer to here is, did removing hardware compression from the data path of the aux copy to tape end up showing the tape filling to compressed capacity?  I am seeing the same thing where the tape is filling to native capacity rather than compressed capacity, and would like to know if unchecking compression on the Simpana side did the trick..?

    All your base are belong to us....

  • Re: Auxcopy To Tape Speed - Hardware Compression?
    Posted: 06-26-2017, 9:22 AM

    Any updates here?

The content of the forums, threads and posts reflects the thoughts and opinions of each author, and does not represent the thoughts, opinions, plans or strategies of Commvault Systems, Inc. ("Commvault") and Commvault undertakes no obligation to update, correct or modify any statements made in this forum. Any and all third party links, statements, comments, or feedback posted to, or otherwise provided by this forum, thread or post are not affiliated with, nor endorsed by, Commvault.
Commvault, Commvault and logo, the “CV” logo, Commvault Systems, Solving Forward, SIM, Singular Information Management, Simpana, Commvault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, SnapProtect, ROMS, and CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.
Copyright © 2018 Commvault | All Rights Reserved. | Legal | Privacy Policy