Media Agent recovery.

Last post 03-19-2019, 7:32 AM by antares. 5 replies.
Sort Posts: Previous Next
  • Media Agent recovery.
    Posted: 03-15-2019, 9:40 AM

    Hello,

     

    i am getting familiar with CommVault (v11 SP13) and while looking through forum found some options to recover Media agent after crash. However 1-Touch wont work for me because we are backing up OS fia File System plus Indexes via IndexBackup client that is created for every storage policy. 

    Storage policy that holds baclups of Media agent and Indexes has local copy and secondary is sent to main data center. In case we lose whole Media agent with drives, recovery would be:

    - Install new server with the OS;

    - Install Media Agent package;

    - Resotore Indexes from secondary copy;

    - Restore DDB from secondary copy;

    - Reconnect Disk Library;

    - Index cahce will be re-created when new jobs comes in;

    I am curious about Index'es. If restoring them from secondary copy is the only way to get them back?

    Does it make sense?

    Thanks.

  • Re: Media Agent recovery.
    Posted: 03-15-2019, 7:47 PM

    Hi antares

    In a disaster scenario where your Primary MediaAgent crashes and needs to be rebuilt, majority of the steps are correct:

    • Install new server with the OS;
    • Install Media Agent package;
    • Assuming the Storage is a separate SAN/NAS, then all you need is to connect the LUNs or ensure the UNC are accessible.
    The other two components DDB and Index the process should be like
     
    • Assuming DDB Backup is still available from the Primary Data Center, you will run a DDB Recon to recover the DDB to the state when it crashed 
      • Note - even if you don't have a DDB backup you can still trigger a Full DDB Recon which will read the metadata reference from the Disk Library (assuming still healthy) to rebuild the DDB
    • Index Cache, you don't need to rebuild, because when you do a Backup it will automatically create new Index otherwise it will replay the index metadata from the Disk Library.
      • Same goes for "Browse & Restore" if the Index does not exist it will replay back from the Disk Library
     
    Technically only the Media Agent OS, Binary and DDB needs to be manually rebuilt, all the other setting (are held by the CSDB) and component will still be on the Disk Storage.
     
    Hopefully that explains and covers everything 
     
    Feel free to reach out if you have any other questions 
     
    Regards
     
    Winston 
  • Re: Media Agent recovery.
    Posted: 03-18-2019, 6:35 PM

    Thanks a bunch Winston! Your post explains a lot. My only concern is when we lose the whole thing - Media agent along with disk libraries (DDB and Index reside on disks inside the server). In this scenario, everything needs to be recovered from the main site?

    DDB needs to be recreated from DDB backup, same as Index? 

    Thanks.

  • Re: Media Agent recovery.
    Posted: 03-19-2019, 6:34 AM

    Hi Antaries 

    In the worst scenario possible, where you lose:

    • MediaAgent 
    • DDB 
    • Index 
    • & Disk Library (SAN)
    Then the challenge and what you will need to ensure is whether your Secondary Copy, is up to date (in your Primary Data Center) 
     
    At this point you won't be able to rebuild the DDB or Index anymore, but will need to Aux Copy everything back from your Secondary Site
     
    E.g. (to give you an example)
    • Site A (MA 1, DDB 1 and Disk 1) 
    • Site B (MA 2, DDB 2 and Disk 2) -> Main Data Center
    Assuming the two sites replicate to each other, Site is Primary and Site B is Secondary, both site has same retention. 
     
    In the worst case scenario where Site A is lost (everything), you will first rebuild:
    • MA,
    • Provision the DDB & Index Volume 
    • Re-create the Mount Paths (assuming it is complete gone)
    Steps:
    Once the MA on Site A is rebuilt, since you have no data to rebuild the DDB, you will instead Seal the DDB (in Site A), cleanup/fix up the Mount Paths, so everything is freed up and have no more association. For that specific Storage Policy you will then Promote Site B as the Primary Copy and run an Aux Copy which will seed the data back to the new MA in Site A. 
     
    Its a very horrible situation to be in, but that is why it is important to have a Secondary Site set correctly, and the on-going Aux Copy is up to date, so you have redundancy even if an entire site is lost. (this will ensure redundancy beyond the individual hardware level, but at Site Level).
     
    Hopefully that helps and provides more context
     
    Kind Regards 
     
    Winston
  • Re: Media Agent recovery.
    Posted: 03-19-2019, 7:12 AM

    I would recommand to every Customers to have a synchronous Copy from the Primary Disk Data on Secondary Device for minimum 1 Cycle. This is much importent, to have to possability to seal the DDB Store in Event of loosing both, DDB Disks and Magnetic Library. You can then restore the environment with the backups of all clients in the backhand.

    The more Common case is simple to lose the DDB Disk and have no DDB-Backup available or the Rebuild takes to long time. Sometime Rebuild with the last DDB-Backup is quicker, sometimes reading the entire Disk library is also fast, but realy dependance from the environment.

    In this Case most Customer do not have enough Space on there Disklibrary for a new Baseline (new deuplicated initial Full from all clients) and run out of space. If you have a secondary Synchronous Copy you can decrease or maybe age all primary Jobs to get enough Space.

  • Re: Media Agent recovery.
    Posted: 03-19-2019, 7:32 AM

    Thank you guys! Appreciate your input, this is very helpful!

The content of the forums, threads and posts reflects the thoughts and opinions of each author, and does not represent the thoughts, opinions, plans or strategies of Commvault Systems, Inc. ("Commvault") and Commvault undertakes no obligation to update, correct or modify any statements made in this forum. Any and all third party links, statements, comments, or feedback posted to, or otherwise provided by this forum, thread or post are not affiliated with, nor endorsed by, Commvault.
Commvault, Commvault and logo, the “CV” logo, Commvault Systems, Solving Forward, SIM, Singular Information Management, Simpana, Commvault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, SnapProtect, ROMS, and CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.
Close
Copyright © 2019 Commvault | All Rights Reserved. | Legal | Privacy Policy