Garbage collection

Last post 08-10-2020, 12:34 AM by Flipper. 10 replies.
Sort Posts: Previous Next
  • Garbage collection
    Posted: 02-06-2020, 8:07 PM

    How is aging with Garbage collection different than without garbage collection? Does it not keep zero reference table?


    Saurabh Srivastava
  • Re: Garbage collection
    Posted: 02-09-2020, 8:05 AM

    Hi for.an.example 

    With Garbage Collection the DDB disk is only parsed once every 24 hours to mark the CommServe job records for pruning and the data aging process.

    The garbage collection uses a bitmap file to collect information about what needs to be pruned.  The change is Commvault add entries into the zero ref table (which is still required)

    This process puts less load on the DDB, whereas the traditional DDB process needs to decrement the reference whenever it is not needed in the DDB

    Regards

    Winston

     


  • Re: Garbage collection
    Posted: 02-23-2020, 12:48 PM

    Hi Winston,

    That answer well. Thank you for clarifying this.

    Regards

    Saurabh


    Saurabh Srivastava
  • Re: Garbage collection
    Posted: 02-23-2020, 12:48 PM

    Hi Winston,

    That answers well. Thank you for clarifying this.

    Regards

    Saurabh


    Saurabh Srivastava
  • Re: Garbage collection
    Posted: 08-04-2020, 8:53 AM

    Hi,

    Is it possible to force the garbage collection immediately ?

    Without waiting for 24 hour interval.

    For example to free space on disk library ASAP.

    Thanks

    Jiri

  • Re: Garbage collection
    Posted: 08-06-2020, 12:10 AM

    Hi Jiri,

    Don't know about whether it is possible to force a garbage collection task on-demand, but wanted to mention the Tech talk video prepared by Carl Brault named, "Deduplication Generation 4 Version 2", which you may find helpful as it clarifies the difference between the older and newer DDB version, especially as regards the benfits around pruning of records and garbage collection which the newer DDB version provides.

    Video is available at https://commvaultondemand.atlassian.net/wiki/spaces/ODLL/pages/501350567/Tech+Talks (may require Commvault Education Adfantage subscription/log-on).

    Regards,

    Michael

  • Re: Garbage collection
    Posted: 08-06-2020, 1:50 AM

    Hi Michael,

    Thanks for the video link. I have seen it already.

    My question is maybe more general. How to free capacity on disk library (with deduplication) ASAP in case when there is low free space.

    We usually do:

    1. Delete some old/unnecessary jobs on copy or shorten retention for some storage policy
    2. Run data aging manually (several times)
    3. Wait till space is released by pruning (sometimes hours ??)

    If i understand it correctly blocks are pruned when they are in zeroref table in the DDB.

    The blocks are placed to zeroref table by garbage collection which runs once per 24 hour. So in the worst case we will be  waiting for 24 hours for getting some free capacity.

    Do you (or anybody else) know how to speed up the process ? Manually force to run garbage collection ? Have triggered garbage collection for "low disk space" event automatically ?

    Thanks

    Jiri

  • Re: Garbage collection
    Posted: 08-06-2020, 2:46 AM

    Hi Jiri,

    Alas, apart from the same options you listed, deleting old jobs, reducing copy retention and running a data-aging job, I have not found any other way to reclaim space on the DDB partition/volume, much less to speed-up the process, and none of these are ideal since you are effectively reducing the restore-points etc.

    Just looked again and I have found a setting in the Control Panel > Media Management > Service Configuration tab labelled, "DDB disk space chek interval in minutes" which has a value of "1440" which equates to once-a-day (24 hours).  Theoretically you could lower this to "720" to run twice-a-day, though I would only do this after checking with support on whether this is supported, or a bad idea due to the potential overhead  it may incur to the Commcell.

    There is also a "Deduplication" tab in Media Management Control Panel dialog, but I do not know that any of the settings in this tab would be of use for what you want.

    The other option aligned to this is to adjust the built-in Data Aging Schedule Policy to run more frequently to deal with DDB and disk library space issues, but again, I do not know that this is a good practice, let alone sustainable or as effective as lowering backup copy retention or manually deleting old jobs and then waiting for the regular 12 PM data-aging job to run just once.

    Just saw there is also the built-in "System created DDB Space Reclammation schedule policy" in the Schedule Policies which appears to be the most simple/relevant option, however, I am not entirely sure this Schedule Policy is tied to "garbage collection" or pruning etc.  I presume that if you are on SP 14> and ran a compaction of the DDB to make it compatible with the new DDB that supports garbage collection, then this schedule may be relevant, but support would likely be able to confirm this (we currently do not have this schedule policy enabled since we have not yet had a chance to compact all our DDB engines try it out.

    Regards,

    Michael

  • Re: Garbage collection
    Posted: 08-08-2020, 3:36 AM

    Hi Flipper,

    FYI, DDB Space Reclamation is only for Storage Targets that do not support the Dedupe Drill Holes technology (e.g CV HS Gluster File System).

  • Re: Garbage collection
    Posted: 08-08-2020, 12:43 PM

    Hi Flipper,

    I investigated SIDBxxxx.log files a little and i have some maybe useful outcomes:

    It seems, that the Garbage Collestion is named "Mark and Sweep" inside of SIDBEngine.log file.

    As an example you can look for string "StartMSThread" in the log file.

    I have searched through documentation and found qscript SetDDBMarkAndSweepInterval, by which you can change default 24 hours interval for particular DDB.

    I tested changing one of my DDB settings (to 12 hours) and it works. This time I looked for string "MS run interval" in SIDBEngine.log  to see change from default 86400 secs to 43200 secs for modified DDB.

    Regards

    Jiri

  • Re: Garbage collection
    Posted: 08-10-2020, 12:34 AM

    Hi Anthony,

    Thank for the pointers. 

    Jiri, that was great detective-work.

    Regards,

    Michael

The content of the forums, threads and posts reflects the thoughts and opinions of each author, and does not represent the thoughts, opinions, plans or strategies of Commvault Systems, Inc. ("Commvault") and Commvault undertakes no obligation to update, correct or modify any statements made in this forum. Any and all third party links, statements, comments, or feedback posted to, or otherwise provided by this forum, thread or post are not affiliated with, nor endorsed by, Commvault.
Commvault, Commvault and logo, the “CV” logo, Commvault Systems, Solving Forward, SIM, Singular Information Management, Simpana, Commvault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, SnapProtect, ROMS, and CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.
Close
Copyright © 2020 Commvault | All Rights Reserved. | Legal | Privacy Policy