Hyper V VM backup holding Partial Success Status

Last post 03-14-2017, 11:48 AM by Jan Rezab. 18 replies.
Sort Posts: Previous Next
  • Hyper V VM backup holding Partial Success Status
    Posted: 12-07-2016, 4:15 AM

    Hello All,

    We are backing up VM's using Hyper V on commvault. We are seeing below errors on most of the VM's  backup status. Most of Hyper V VM backup holding Partial Success Status


    Server Name\\\?\GLOBALROOT\Device\CSV{a10681e0-3595-4c49-a442-b41235dcc85a}\Server Name\Server Name\Snapshots\AEF70BF8-9663-4B07-8485-2F92EE4A0681.xml Failed Virtual machine [Server Name] configuration file [\\?\GLOBALROOT\Device\CSV{a10681e0-3595-4c49-a442-b41235dcc85a}\Server Name\Server Name\Snapshots\AEF70BF8-9663-4B07-8485-2F92EE4A0681.xml] is missing or inaccessible, hence skipped in this backup, Server Name\C:\ClusterStorage\volume3\Server Name\virtual hard disks\sr05020_c-AutoRecovery.avhdx Failed Virtual hard disk [sr05020_c-AutoRecovery.avhdx] attached to virtual machine is missing or inaccessible, hence skipped in this backup, Server Name\C:\ClusterStorage\volume3\Server Name\virtual hard disks\Server Name_D-AutoRecovery.avhdx Failed Virtual hard disk [Server Name_D-AutoRecovery.avhdx] attached to virtual machine is missing or inaccessible, hence skipped in this backup

     

    If possible, please help me to resolve this issue.

  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 02-16-2017, 10:35 AM

    We're experiencing this same issue after upgrading from v10 to v11 sp6 with all hotfixes.  Haven't found a solution as of yet either.

  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 02-17-2017, 6:27 AM

    Hello,

    If you initiate the backup using Crash consistent then backup will be completed. In my infra, Crash consistenmt is not allowed hence we are running backups on Application Coonsistent mode.

  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 02-20-2017, 10:45 PM
    • Ali is not online. Last active: 05-31-2017, 2:12 PM Ali
    • Top 10 Contributor
    • Joined on 08-05-2010

    Hi Rohit, have you logged an incident for this?

  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 02-21-2017, 1:56 AM

    Yes, Many times i have logged a case for this. Initially, our Hyper-V host and VM were lacking the latest windows updates. Due to this, Commvault suggested us to patch the Hyper V hosts to the latest level, we did the same, since i dont see any improvement.

    Recommandation which Commvault gave, i didnt beleive in that because, Several VM's are running on the same host and 99% are running fine. I have issue with only 2 or 3 VM's hence this isolates the issue which is being addressed towards the patch Levels .

     

     

  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 02-21-2017, 12:43 PM
    Rohit,
     
    It appears that 2 or 3 virtual machines which are showing this behaviour could be hitting a VSS issue
    where they are unable to create internal VSS shadow. Could you please check the hosting Hyper-V 
    server VMMS logs when the backup activity runs to check any errros here. Once we have verified the errors  
    we can correct the VSS failure or disable Backup (Volume checkpoint) integeration service on selected VMS 
    which are noticing this behaviour.

    Please let us know if this helps
  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 02-23-2017, 3:23 AM

    I ran backup again and checked the VMMS logs "

     

    VSS writers inside virtual machine 'SR50119' failed to perform BackupComplete to its shadow copy (VSS snapshot) set: Catastrophic failure (0x8000FFFF). (Virtual machine ID 4E49D9D5-BA4A-4910-B11C-7E48400F3871)

    Checkpoint operation for 'SR50119' was cancelled. (Virtual machine ID 4E49D9D5-BA4A-4910-B11C-7E48400F3871)

    Checkpoint operation for 'SR50119' failed. (Virtual machine ID 4E49D9D5-BA4A-4910-B11C-7E48400F3871)

     

    I have also tested system state backup using  Windows System backups utility from within the VM to check if there is any issue in VM itself, however system state backup went fine with-in the VM.

    Thanks,

    Rohit Sharma

  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 02-23-2017, 11:40 AM

    The error points that VSS inside the guest VM is failing to create a consistent snapshot of the guest VM. Could you please try disabling Backup (Volume checkpoint) service on the guest VM.

    Once the VSS resolved inside the guest VM you could reenable this integeration service again.

  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 02-23-2017, 12:31 PM

    There is no VSS issue inside the guest VM as we have already check system state backup of guest VM and that went fine.

  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 02-23-2017, 4:59 PM

    The system state backup engages only the ASR writer and system writer. In this case I beleive we are running an application consistent backup so all eligible writers inside the guest VMs are gettting engaged and it appears that one of the writer could be failing.

    Disabling backup volume check point integeration service will help here to overcome the VSS failures inside the guest VM.

  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 02-26-2017, 7:42 AM

    Hello Sanjay,

    You are right, we are running File System and Application Consistent backups. I followed the instruction given by you and disabkled backup volume check point integeration service for VM's using Hyper V Manager and it helped me to resolve the issue.

    I appreciate your help on this. Thank you very much :)

    I just have small question, after Disabling backup volume check point integeration service, are backup still being committed as File System and Application Consistent backups ?

     

  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 02-27-2017, 10:14 AM

    Hello Sanjay,

    I checked this with Commvault and told them that we disabled the backup volume check point integeration service and after that backup went fine, however they confirmed that if we disable the backup volume check point integeration service then it would be creash consistent backup rather than File and Application consistent.

    Is there any other way we can achive the successful completion of these VM with application consistent backup.

    Thanks,

    Rohit Sharma

  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 02-27-2017, 11:32 AM

    Rohit,

    Good to know that the backups are completing successfully. Could you please attempt to create a VSS shadow inside the guest VM and see if VSS operations are completing successfully.

    Please refer to examples section of below link. 

    https://technet.microsoft.com/en-us/library/cc772172(v=ws.11).aspx

    Thanks

    Sanjay

     

     

  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 02-27-2017, 2:32 PM

    Rohit,

    As we have disabled backup volume check point integration service on the guest VM, the Hyper-V writer on the host ensures that VSS infrastructure inside the guest VM is not engaged during the VSS check point operation. So the backup snapshot/checkpoint created on the guest VM is not application consistent, however the snapshot is still file system consistent and backup has protected that state of VM.

    To turn this integration service turn on we need to first address the VSS issue inside the guest VM. Once this is done the option can be re-enabled at guest level.

    Please follow the steps as outlined in the Microsoft article and see if we are seeing any VSS error in guest.

    Thanks

    Sanjay

     

  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 03-02-2017, 7:15 AM

    Hello Sanjay,

    This article has so many component, if possible can you please let me know what component of DiskShadow needs to be implemented.

    Thanks,

    Rohit Sharma

     

  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 03-14-2017, 6:13 AM
    I have the same issue. 11 SP6 + all hotfixes.
  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 03-14-2017, 11:40 AM

    Jan,

    Could you please share the actual error message shown in the "Failure Reason" column. In addition what type of backups we are running here. Are they crash-consistent or application consistent.

    Thanks

    Sanjay

  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 03-14-2017, 11:47 AM
    VM_HSP1\\\?\GLOBALROOT\Device\CSV{c836d999-6efd-4f85-acf2-2280e2095f0a}\Snapshots\FCD5BB73-7535-49D3-8A15-C014409FABB7.xml Failed Virtual machine [VM_HSP1] configuration file [\\?\GLOBALROOT\Device\CSV{c836d999-6efd-4f85-acf2-2280e2095f0a}\Snapshots\FCD5BB73-7535-49D3-8A15-C014409FABB7.xml] is missing or inaccessible, hence skipped in this backup VM_HBA1\\\?\GLOBALROOT\Device\CSV{c836d999-6efd-4f85-acf2-2280e2095f0a}\Snapshots\08B3A5FF-7AB8-41D1-B284-5F91CB4A0C36.xml Failed Virtual machine [VM_HBA1] configuration file [\\?\GLOBALROOT\Device\CSV{c836d999-6efd-4f85-acf2-2280e2095f0a}\Snapshots\08B3A5FF-7AB8-41D1-B284-5F91CB4A0C36.xml] is missing or inaccessible, hence skipped in this backup HFTP1\\\?\GLOBALROOT\Device\CSV{c836d999-6efd-4f85-acf2-2280e2095f0a}\Snapshots\0CC7506B-2AD0-4AB0-8D08-604C632D72C8.xml Failed Virtual machine [HFTP1] configuration file [\\?\GLOBALROOT\Device\CSV{c836d999-6efd-4f85-acf2-2280e2095f0a}\Snapshots\0CC7506B-2AD0-4AB0-8D08-604C632D72C8.xml] is missing or inaccessible, hence skipped in this backup I use File System and Application Consistent.
  • Re: Hyper V VM backup holding Partial Success Status
    Posted: 03-14-2017, 11:48 AM
    VM_HSP1\\\?\GLOBALROOT\Device\CSV{c836d999-6efd-4f85-acf2-2280e2095f0a}\Snapshots\FCD5BB73-7535-49D3-8A15-C014409FABB7.xml Failed Virtual machine [VM_HSP1] configuration file [\\?\GLOBALROOT\Device\CSV{c836d999-6efd-4f85-acf2-2280e2095f0a}\Snapshots\FCD5BB73-7535-49D3-8A15-C014409FABB7.xml] is missing or inaccessible, hence skipped in this backup VM_HBA1\\\?\GLOBALROOT\Device\CSV{c836d999-6efd-4f85-acf2-2280e2095f0a}\Snapshots\08B3A5FF-7AB8-41D1-B284-5F91CB4A0C36.xml Failed Virtual machine [VM_HBA1] configuration file [\\?\GLOBALROOT\Device\CSV{c836d999-6efd-4f85-acf2-2280e2095f0a}\Snapshots\08B3A5FF-7AB8-41D1-B284-5F91CB4A0C36.xml] is missing or inaccessible, hence skipped in this backup HFTP1\\\?\GLOBALROOT\Device\CSV{c836d999-6efd-4f85-acf2-2280e2095f0a}\Snapshots\0CC7506B-2AD0-4AB0-8D08-604C632D72C8.xml Failed Virtual machine [HFTP1] configuration file [\\?\GLOBALROOT\Device\CSV{c836d999-6efd-4f85-acf2-2280e2095f0a}\Snapshots\0CC7506B-2AD0-4AB0-8D08-604C632D72C8.xml] is missing or inaccessible, hence skipped in this backup I use File System and Application Consistent.
The content of the forums, threads and posts reflects the thoughts and opinions of each author, and does not represent the thoughts, opinions, plans or strategies of Commvault Systems, Inc. ("Commvault") and Commvault undertakes no obligation to update, correct or modify any statements made in this forum. Any and all third party links, statements, comments, or feedback posted to, or otherwise provided by this forum, thread or post are not affiliated with, nor endorsed by, Commvault.
Commvault, Commvault and logo, the “CV” logo, Commvault Systems, Solving Forward, SIM, Singular Information Management, Simpana, Commvault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, SnapProtect, ROMS, and CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.
Close
Copyright © 2017 Commvault | All Rights Reserved. | Legal | Privacy Policy