Process partinfo causes backups to stall at 80%

Last post 05-15-2018, 1:07 PM by jhshen. 4 replies.
Sort Posts: Previous Next
  • Process partinfo causes backups to stall at 80%
    Posted: 05-09-2018, 7:26 PM

    My 1-Touch servers have daily incremental backups and weekly full backups. But currently, all backups simply hang at 80%. Investigation shows that the cause is the partinfo command. Doing a command like:

    # ps Hax | grep partinfo

    You may see many processes:
    10607 ? S 0:19 /opt/simpana/iDataAgent/systemrecovery/partinfo disks 2>/dev/null
     
    Have you ever encountered this problem before and do you know the resolution?


  • Re: Process partinfo causes backups to stall at 80%
    Posted: 05-10-2018, 11:21 AM

    Hello jshen,

    This is likely an indication your server currently has some disk(s) having I/O errors. 

    1. Could you give us the output of : "lsblk" from this server? 

    2. Is rebooting the server an option?

     

    thanks,

    Sumedh

     

  • Re: Process partinfo causes backups to stall at 80%
    Posted: 05-10-2018, 11:42 AM

    It may not be a specific server issue because 12 servers have this problem. And it's unlikely that all 12 servers have the same problem.

     

    1. Unfortunately running the lsblk seemed to stall after waiting for 15 minutes with no results. I tried to kill the process and the process did not stop. I had to do a reboot of the server.

    2. Rebooting the server or restarting the process did not fix the problem. It simply stalled again at 80%.

  • Re: Process partinfo causes backups to stall at 80%
    Posted: 05-10-2018, 12:31 PM

    Do you have SAN/NAS devices connected to these servers?

    As a first step, I would recommend uncheck the "1-Touch recovery" option so as to continue protecting your servers with your daily incremental FS backups via the File System Agent. This will ensure your postbackup phase will not run, and your backups complete, albeit without the ability to recover using 1-Touch.

    Since you mention "lsblk" was also stuck, that indicates this issue is related to one or more disks having I/O errors. i.e. partinfo stalled for the same reason as the OS utility "lsblk" stalled.

    You can use this script to identify the problematic device. note that the command will get stuck, but should have printed the device name before it gets stuck, so you can identify the problematic disk and take necessary action.

    # for dv in `ls /sys/block`; do echo "check $dv"; lsblk /dev/$dv; echo "";  done

    -Sumedh

  • Re: Process partinfo causes backups to stall at 80%
    Posted: 05-15-2018, 1:07 PM

    Updating the current clients to the latest OS SUSE Linux 11 SP4 seemed to fix the problem. The 1-Touch Backups seem to work fine now.

The content of the forums, threads and posts reflects the thoughts and opinions of each author, and does not represent the thoughts, opinions, plans or strategies of Commvault Systems, Inc. ("Commvault") and Commvault undertakes no obligation to update, correct or modify any statements made in this forum. Any and all third party links, statements, comments, or feedback posted to, or otherwise provided by this forum, thread or post are not affiliated with, nor endorsed by, Commvault.
Commvault, Commvault and logo, the “CV” logo, Commvault Systems, Solving Forward, SIM, Singular Information Management, Simpana, Commvault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, SnapProtect, ROMS, and CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.
Close
Copyright © 2018 Commvault | All Rights Reserved. | Legal | Privacy Policy