Postgres backup for 20TB database

Last post 08-29-2019, 7:01 AM by Meera. 8 replies.
Sort Posts: Previous Next
  • Postgres backup for 20TB database
    Posted: 06-04-2019, 2:58 PM
    I'm trying to set up backup for 20TB postgres database. According:

    http://documentation.commvault.com/commvault/v11/article?p=21532.htm

    we are not able to setup multiple streams or multiple data readers (like in filesystem client). Result i get is around 200GB/hr (same as with filesystem backup with one data reader). Any way to improve that?


    As FSBased backup based on standard file backup why we are not able tune it same way and it need to use one stream?
  • Re: Postgres backup for 20TB database
    Posted: 06-04-2019, 3:37 PM
    • efg is not online. Last active: 10-22-2019, 2:17 PM efg
    • Top 10 Contributor
    • Joined on 02-02-2010
    • CommVault Tinton Falls NJ
    • Master
    • Points 1,678

    Check to see if perhaps you can enable block level backup for your postgres instance.  With block level backup you can then run multi-stream backup and restore.  You need to make sure you are on V11SP15 (minimum) for this to work though.

    Here is the link for PostGres with Block Level backup:

    PostgreSQL Block-Level Backup


    Ernst F. Graeler
    Senior Engineer III
    Development
  • Re: Postgres backup for 20TB database
    Posted: 06-04-2019, 4:28 PM

    efg:

    Check to see if perhaps you can enable block level backup for your postgres instance.  With block level backup you can then run multi-stream backup and restore.  You need to make sure you are on V11SP15 (minimum) for this to work though.

    Here is the link for PostGres with Block Level backup:

    PostgreSQL Block-Level Backup

    Well, not an option. Database is put on non lvm volume. Also OS is Ubuntu (not supported). But if there is no other way, we may need to consider migration. 

    SP15 is a must? Any bugs in earlriler releases?

    Any reason why fs based postgres backup can't use multipple readers?

     

    Cheers.

  • Re: Postgres backup for 20TB database
    Posted: 06-04-2019, 11:56 PM

    Hi, 

    We do not support multistream for streaming backups yet. A CCR will help to consider it on priority. 

    WE support multiple stream for backups with snap and block level in v11 .

    For block level:

    Till sp15: indexing v1 type is only supported which supports multiple stream for backup but not for restore.

    With sp15: Indexing v2 is also supported. This supports multistream backup and restore.

    On ubuntu, they have an option to run driverless block level backup. 

    https://documentation.commvault.com/commvault/v11/article?p=93187.htm

    With driverless they can take full backups and restores. Incremental, synth full backup and table level restore is not supported. 

     

    What is the kernel version and exact ubuntu version?

    If File system iDA supports we can check for the support. 

    Ad you mentioned its non lvm volume. What is the configuration? Is it coming from a supported array? We support backup for local device also provided free extents are available

     

    Regards,

    Meera 

  • Re: Postgres backup for 20TB database
    Posted: 06-05-2019, 2:44 AM

    Slightly off topic, but can someone clarify what Commvault mean when it stated that with FSBased Backup Sets "You cannot perform redirect restore operations on PostgreSQL database block-level backups" https://documentation.commvault.com/commvault/v11_sp15/article?p=21585.htm.  How is a directed restore no different to what has been described here https://documentation.commvault.com/commvault/v11_sp15/article?p=61285.htm?

  • Re: Postgres backup for 20TB database
    Posted: 06-05-2019, 3:38 AM

    Hi Anthony,

    Few changes are needed in doc with the changes made for sp15.

    Cross machine restore used to restore data directory and archive log directory to the path same as that of source till we introduced redirect restore feature. 

    Redirect restore is an option in Advanced restore dialog which allows user to redirect the data directory and tablespace directories to paths different from that of the source. And archive log directory (and port) are taken from the destination instance properties.

    WE will update the BOL with sp16 release

     

    Regards,

    Meera 

  • Re: Postgres backup for 20TB database
    Posted: 06-05-2019, 4:05 AM

    Meera:

    What is the kernel version and exact ubuntu version?

    If File system iDA supports we can check for the support. 

    Ad you mentioned its non lvm volume. What is the configuration? Is it coming from a supported array? We support backup for local device also provided free extents are available

    Ubuntu 18.04.2 LTS, with 4.15.0-X. Looks like Ubuntu is not supported for block level backup for filesystem iDA. 

    Yes, storage is provided by supported snap engine array. Another option to consider.  

     

    Thanks for explanation. 

  • Re: Postgres backup for 20TB database
    Posted: 08-29-2019, 4:55 AM

    Another question. With driverless, is restore from backup copy supported?

    I'm able to restore from primary (snapshot copy). But when i move snapshot to disk library (using backup copy) restore failed. Client need to be able to mount pseudomountpoint, so i assume driver is reqiured? Choosing media agent to restore with cvblk driver loaded also ends with error (wrong mountpath). So wondering if such configuration is supported at all or if it's a bug. 

     

     

     

  • Re: Postgres backup for 20TB database
    Posted: 08-29-2019, 7:01 AM

    Restore is supported.

    Please try using a proxy which has a supported kernel to perform restores 

    In restore dialog -> advanced tab -> Proxy has to be selected

    Supported kernels can be found here

    http://documentation.commvault.com/commvault/v11_sp16/article?p=3515_1.htm

The content of the forums, threads and posts reflects the thoughts and opinions of each author, and does not represent the thoughts, opinions, plans or strategies of Commvault Systems, Inc. ("Commvault") and Commvault undertakes no obligation to update, correct or modify any statements made in this forum. Any and all third party links, statements, comments, or feedback posted to, or otherwise provided by this forum, thread or post are not affiliated with, nor endorsed by, Commvault.
Commvault, Commvault and logo, the “CV” logo, Commvault Systems, Solving Forward, SIM, Singular Information Management, Simpana, Commvault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, SnapProtect, ROMS, and CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.
Close
Copyright © 2019 Commvault | All Rights Reserved. | Legal | Privacy Policy