Optimizing Aux Copy Over Wan

Last post 08-22-2013, 5:34 AM by dandre. 8 replies.
Sort Posts: Previous Next
  • Optimizing Aux Copy Over Wan
    Posted: 07-26-2013, 2:28 PM

    We have (2) datacenter's geographically spread out with a 1GB Wan link latency is 53ms (Expected for that distance).  We are getting 5MByte/Sec throughput when set to HTTP tunnel in commvault, that drops to 1.5MByte/Sec when using https tunnel in commvault.  We have utilized fortinet wan optimization and saw improvements to 35 MByte/Sec for many types of tcp traffic (SMB/CIFS) however no improvement for commvault.  This did not matter if we used dedupe or non-dedupe, has anybody use any other wan optimization tools and had success, or is there any other configuration to look for?

  • Re: Optimizing Aux Copy Over Wan
    Posted: 07-29-2013, 11:31 PM
    • Ali is not online. Last active: 07-03-2019, 12:32 PM Ali
    • Top 10 Contributor
    • Joined on 08-05-2010

    Hi Technoz, hope you are well.

    Have you had a look here by chance, just to clarify this is a DeDupe enabled Disk to Disk DASH Aux copy?

    http://documentation.commvault.com/commvault/release_10_0_0/books_online_1/english_us/prod_info/dedup_disk.htm?var1=http://documentation.commvault.com/commvault/release_10_0_0/books_online_1/english_us/features/dash_copy/performance_tuning.htm

  • Re: Optimizing Aux Copy Over Wan
    Posted: 08-10-2013, 3:22 AM

     

    If you are using Dash Copy and the baseline has been written to the destination dedupe database, for best performance use Disk Optimized.

    You may also like to consider using Look-Ahead Reader. 

    To reduce the time taken to read the data during restore and auxiliary copy operations, deduplication-enabled operations can be performed using look-ahead reader. 0/books_online_1/english_us/features/dedup_disk/advanced.htm#Look-Ahead_Logic

    Laughing
  • Re: Optimizing Aux Copy Over Wan
    Posted: 08-17-2013, 1:01 AM

    I stumbled across a fix..Under the media agent properies I unchecked "Optimize for concurrent lan backups" I went from 18GB/HR with optimize checkmarked, to 225GB/HR with optimize off.  Removing optimize sped up all my dash copies immensily.  Only problem I have now with making that change is my local sql backups are taking 15% longer.

  • Re: Optimizing Aux Copy Over Wan
    Posted: 08-19-2013, 8:26 AM

    Hello technoz,

    Presume you're on v10.  There's an update available that addresses this symptom where auxcopy using "Optimize for concurrent LAN backups" on MAs runs into a slowness issue.

    Please see if the symptom persists after installing that update on the source and destination MAs and keeping the 'optimize..." option on.  Suggest you call into support about this so you can have that update made available on top of sp3a.

    Thanks

     Jaidev.

  • Re: Optimizing Aux Copy Over Wan
    Posted: 08-19-2013, 12:01 PM

    Actually running V9 SP10 :( Waiting for support to call me back to see whats up.

  • Re: Optimizing Aux Copy Over Wan
    Posted: 08-21-2013, 2:16 PM

    Hi Technoz,

    Optimize for concurrent lan backups is a more a efficient way of transferring data. I think it was introduced around the V6/V7 timeframe. However there is one caveat.

    When you use a firewall configuration, it will not make use of the 'additional ports'. This means that your data transfer IS going over the tunnel port, and the tunnel port only. Bundling up the data with the HTTP/HTTPS protocol will slow down the data transfer, as there is a lot of overhead when pushing this via the tunnel.

    When you turn off 'optimize for concurrent lan backups' we fall back to the legacy transfer method, which is less efficient on MA resouces, but it will actually use the additional ports you specify in the firewall config.

    This means the data transfer portion of the job is going directly to the CVD process on the remote media agent, which is why your dash copies fly.

    You can actually get the transfer to use one port that is NOT the tunnel port when you have the optimize option turned on. To do this, add the CVD port to the additional ports (in 99% of cases this is port 8400). Make sure that 8400 is open on your firewall between your dash media agents for the transfer to work.

    This is actually documented in BOL:

    http://documentation.commvault.com/commvault/release_9_0_0/books_online_1/english_us/prod_info/features.htm?var1=http://documentation.commvault.com/commvault/release_9_0_0/books_online_1/english_us/features/firewall/firewall_new.htm

    • For MediaAgents with Optimize for concurrent LAN backups option enabled, opening the incoming port of the CommVault Communications Service improves the backup performance.
    So I'd recommend re-enabling the option and adding 8400 as an additional port for the best of both worlds.
  • Re: Optimizing Aux Copy Over Wan
    Posted: 08-21-2013, 3:53 PM

    Thank you Dandre for the excellent explanation, I have ports 8400 - 8441 open on the firewall, so I setup cvlt firewall with listen for tunnel connection on 8441 and under additional ports I added 8400-8440.  I reran the test with optimized enabled and I saw max thruput of 120GB/HR, which is an improvement.  But then I went ahead and disabled optimize and reran and I see thruput of 1.1TB/HR.

  • Re: Optimizing Aux Copy Over Wan
    Posted: 08-22-2013, 5:34 AM

    Interesting!

    If you get a chance, could you run a netstat -nab during data transfer with optimize off and see how many of the additional ports are being used.

    Then turn it on and do the same test.

    As mentioned, only 8400 would be used if you add that in with optimize on. In your environment, perhaps it benefits further from the extra additional ports you have specified as well.

The content of the forums, threads and posts reflects the thoughts and opinions of each author, and does not represent the thoughts, opinions, plans or strategies of Commvault Systems, Inc. ("Commvault") and Commvault undertakes no obligation to update, correct or modify any statements made in this forum. Any and all third party links, statements, comments, or feedback posted to, or otherwise provided by this forum, thread or post are not affiliated with, nor endorsed by, Commvault.
Commvault, Commvault and logo, the “CV” logo, Commvault Systems, Solving Forward, SIM, Singular Information Management, Simpana, Commvault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, SnapProtect, ROMS, and CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.
Close
Copyright © 2019 Commvault | All Rights Reserved. | Legal | Privacy Policy