AWS restore Speeds

Last post 02-11-2020, 9:41 AM by jbyrne123. 6 replies.
Sort Posts: Previous Next
  • AWS restore Speeds
    Posted: 01-31-2020, 10:30 AM

    I am doing some restores in AWS.  I am restoring a full vm (agentless restore).

    I am getting 30GB/Hr for a VM with a single disk drive.  Is this normal?  I get 200GB/Hr doing the same restore in VMWare.

     

    I have a media agent in AWS with SSD for DDB and S3 storage bucket in the same region.  I was expecting the restores to go a little faster.

     

    What do restore rates do people typicaly get when resotoring an EC2 instance? 

     


    Jim Byrne
    Principal Systems Engineer
    DataPivot Technologies Inc.
    jbyrne@datapivottech.com
    203-631-9236 mobile
  • Re: AWS restore Speeds
    Posted: 02-04-2020, 11:47 AM

    Can't speak to restores, but replication from a local vcenter to aws I'm getting ~89GB/Hr.  Where is the backup data being restored located?

  • Re: AWS restore Speeds
    Posted: 02-09-2020, 7:56 AM

    Hi jbryne123

    While doing restore you will need to consider which component might be the bottleneck:

    • Network speed 
    • Read Performance from the S3 Bucket 
    • Write speed back to the Destination
    Depending on the above component there might be additional configurables that could improve performance. 
     
    Regards
     
    Winston
  • Re: AWS restore Speeds
    Posted: 02-10-2020, 8:17 AM

    I have setup a media agent in US-EAST-2.  The media agent is an ec2 instance with 8CPU, 64GB RAM, nvme SSD, AWS S3 standard Storage, and running on linux.  I am using CV11SP18.  Please note I have been using Commvault for several years and am aware of the usual bottlenecks.  The test environment is built per commvault's AWS best parctices guide.

    I am not looking for restore speeds for files and databases using an iDA.  I am looking for a restore rate of an EC2 instance in AWS.   For example, I can restore a VM (originally backed up in AWS) to an on-prem VMWare environment, using nbd mode, at 200 to 300GB per hour.

    I think there is a big elephant in the room nobody wants to discsuss.  Restoring EC2 instances in AWS is just plain slow,  this is why everyone replicates EC2 instances or restores from snapshots.

    I have serached on the web for any backup vendors benchmarks regarding restore speeds and nobody will publish any results.  You can find restore rates for VMWare, KVM, and Hyper-V quite easily in various user blogs.

    I would suggest if you have VMs in AWS you should try some restores to set expectations.  If you need to restore a VM, that has a single 1TB disk, it is going to take 30+ hours.

    If anyone has empirical data they would like to share that would be great.  I would like to see what is real and not theoretical.

    Thank You,

    Jim Byrne 

    Principal Systems Engineer 

    DataPivot Technologies Inc. 


    Jim Byrne
    Principal Systems Engineer
    DataPivot Technologies Inc.
    jbyrne@datapivottech.com
    203-631-9236 mobile
  • Re: AWS restore Speeds
    Posted: 02-11-2020, 8:55 AM

    For the restore job, have a look at the performance logs. It will tell you where the bottleneck is, read, write, etc. perhaps share it here. CVPerfMgr.log is a good start.

    Restore jobs do not need DDB, so doesn't matter where the DDB is stored.

  • Re: AWS restore Speeds
    Posted: 02-11-2020, 9:11 AM

    Please note that I am not asking for tuning.  All set with this topic.

    I am looking for actual restore speeds that people have done.  I am looking for sombody who has restored a VM, originally in VMWare, and restored it as a converted EC2 instance up in AWS.

    I supsect most people are taking it for granted that if you backup a VMWare VM on-prem and restore it to AWS as an EC2 instance the process will be quick.  I do not think this is the case.

    I have a customer who backups up and restores EC2 VMs in AWS at 200 to 300GB/Hr.  If he does the same test using an on-prem VM (running in VMWare) the restore is fast at first and then is slow.  There is something about the convsion process that slows down the restore.  Both restores are run from a VM in AWS that has been build per commvault best practices.

    Again, I don't want tuning advice.  I am all set.  I am looking for actual restore restults from my fellow commvault admins.  What speeds are you getting when you restore a VM (backed up from VMWare) and restored to AWS as an EC2 instance?

    Thank You


    Jim Byrne
    Principal Systems Engineer
    DataPivot Technologies Inc.
    jbyrne@datapivottech.com
    203-631-9236 mobile
  • Re: AWS restore Speeds
    Posted: 02-11-2020, 9:41 AM

    Thank you DoctorB for the empirical results!

    The backup data being restored in my POC is in AWS.  We are getting a single restore speed of 30GB/Hour/virtual disk.  If I restore 10 VMs with one virutal disk each, at the same time, I get 300GB/Hr.  The single restore stream seems limited to 30GB/Hour.

     

    My concern is if we have a VM with a 1TB virtual disk then, it will take 30+ hours to restore.

     

    I just want to want to set expectation so we architect a proper solution.  Some large VMs will need to be replciated via LiveSync to meet SLAs.  Small VMs can be restore on the fly.


    Jim Byrne
    Principal Systems Engineer
    DataPivot Technologies Inc.
    jbyrne@datapivottech.com
    203-631-9236 mobile
The content of the forums, threads and posts reflects the thoughts and opinions of each author, and does not represent the thoughts, opinions, plans or strategies of Commvault Systems, Inc. ("Commvault") and Commvault undertakes no obligation to update, correct or modify any statements made in this forum. Any and all third party links, statements, comments, or feedback posted to, or otherwise provided by this forum, thread or post are not affiliated with, nor endorsed by, Commvault.
Commvault, Commvault and logo, the “CV” logo, Commvault Systems, Solving Forward, SIM, Singular Information Management, Simpana, Commvault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, SnapProtect, ROMS, and CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.
Close
Copyright © 2020 Commvault | All Rights Reserved. | Legal | Privacy Policy