Client Side Deduplication Signature Cache questions

Last post 06-09-2011, 3:18 PM by thickburger. 7 replies.
Sort Posts: Previous Next
  • Client Side Deduplication Signature Cache questions
    Posted: 05-25-2011, 12:40 PM

    I have a couple of questions about the Client Side Deduplication signature cache feature:

     

    1) BOL, as well as the online help in the GUI, states: "The signature cache can grow to a maximum of 1 GB."  However, the default size in the GUI (Client >> Properties >> Client Side Deduplication) is set to 4 GB.  Assuming BOL is wrong, what is the "real" max size for the cache?  Is 4 GB the recommended size for the signature cache?  Seems a little large to me.

    2) When Client Side Deduplication is configured globally at the storage policy copy level, my understanding is that ALL subclients associated with the SP will automatically use Client Side Deduplication, even if Client Side Deduplication is not explicitly enabled at the Client >> Properties >> Client Side Deduplication level.  But what about signature caching?  Is it possible to enable signature cachine globally, or must it be enabled individually at the Client >> Properties >> Client Side Deduplication level for each Client?

    3) What are some general rules of thumb for when to / not to use the Client Side Signature Cache?  In my limited testing it hasn't seen like the signature cache provided any major speed improvement compared with just regular client-side dedupe.  But maybe with really large file servers or environments with hundreds of clients it's a good idea to offload as much of the SDB to the clients as possible for greater overall throughput.  Right idea?

  • Re: Client Side Deduplication Signature Cache questions
    Posted: 05-26-2011, 10:38 AM

    1) Max cache size is 32gb (hit the up and down arrow in the GUI).  4 gig is a good size for today's machines assuming the cache is needed.

    2)  Caching shouldnt be used in all cases which is why you dont want this turned on everywhere.  Ill elaborate on point 3.

    3)Client side caching should only be used on WAN links where the lookup on the MA would take too long to efficient process the backup.  The idea here is the local cache would have the sigs so it wouldnt have to hop over the network to the media agent.  Why this isnt good for LAN based is the following:

    Scenario: Say the signature isnt in the local cache but local cache is enabled. 

    The local cache is consulted, the signature isnt there, so it then checks the media agent.  Once it checks the media agents, it then needs to hop back and update the local cache.  This also happens in batches, to optimize network usage (which on the LAN causes further delays)

    If this was on local LAN without the cache option checked, it would simply check the MA and be done with the lookup. 

  • Re: Client Side Deduplication Signature Cache questions
    Posted: 05-26-2011, 11:13 AM

    Thanks!  Great explanation.

  • Re: Client Side Deduplication Signature Cache questions
    Posted: 05-26-2011, 3:14 PM

    Vincenzo that's a very interesting breakdown of the client side caching.

    So if your MA isn't very busy, and the dedupe/sidb drives aren't the quickest, but the MA is on the same gig ethernet LAN as your clients, do you think you would be better or worst off by enabling the option?

  • Re: Client Side Deduplication Signature Cache questions
    Posted: 06-09-2011, 2:16 PM

    Regarding 2) above:

    "When Client Side Deduplication is configured globally at the storage policy copy level, my understanding is that ALL subclients associated with the SP will automatically use Client Side Deduplication, even if Client Side Deduplication is not explicitly enabled at the Client >> Properties >> Client Side Deduplication level."

     

    ...I'm not sure this is the case. 

    On my primary copy for a given storage policy, under Deduplication >> Advanced, the "Enable Deduplication on Clients" box has been checked since I created the primary copy.  I have a given client, and I've NOT enabled Client Side Deduplication on it via Client >> Properties >> Client Side Deduplication.  And, in my Subclient >> Properties >> Storage Device >> Deduplication tab, I've got "Enable Deduplication" checked, and "Generate Signatures on Client" checked.  I've since made several backups, and if client side deduplication were really occurring on this client, an /opt/simpana/iDataAgent/jobResults/CV_CLDB directory should exist for the client side deduplication database.  However, this directory does not exist.  I'm generating signatures on the client, but as best I can tell they are just being packaged up and sent along with all the data to the media agent where it evaluates the signatures and then decides what data is really new and what is already stored in its magnetic library.

    I do find the deduplication terminology in Simpana 9 a bit confusing.

  • Re: Client Side Deduplication Signature Cache questions
    Posted: 06-09-2011, 2:25 PM

    Thickburger:

     

    The "jobResults/CV_CLDB" directory will only exist if the Client-Side Signature Cache feature is enabled.  Client-Side dedupe by itself doesn't create this directory or files.  So, I believe this is what happens is as follows: When you enable Deduplication >> Advanced >> Enable Deduplication on Clients at the storage policy level, ALL clients associated with this storage policy will have client-side dedupe enabled, even if Client-Side Dedupe is not specifically enabled at the Client >> Properties >> Client Side Deduplication checkbox.  However, the local signature cache feature will NOT be enabled unless you specifically enable it at the client level (this was one of my original questions, and has since been confirmed).  This makes sense because we wouldn't want 2-4 GB of space being consumed on all clients all of a sudden.  So in a nutshell, we can enable client-side dedupe globally across all clients associated with a SP, but I don't know of a way to enable signature cache globally (nor would we probably want to, as explained in the posts above).

  • Re: Client Side Deduplication Signature Cache questions
    Posted: 06-09-2011, 2:30 PM

    thickburger, i think this has changed via a patch in the lifecycle of 9.0 so the context of when this statement was made is important, in regards to which option overrides who.

    Now in regards to the CV_CLDB, that database is ONLY created when you have "Enable Client side Disk cache" (thus where the size option comes into play). 

    Signature generation on the client does just that, generates the signatures and sends them to the main DDB.  When you enable the client side disk cache, the signatures are stored within the DB and the main DDB, and the local database is consulted first.  If its not there only then does it send it to the main DDB.

     

     

  • Re: Client Side Deduplication Signature Cache questions
    Posted: 06-09-2011, 3:18 PM

    boylerscs and Vincenzo:

     

    Thanks for your replies, both very helpful at clearing up some things.

The content of the forums, threads and posts reflects the thoughts and opinions of each author, and does not represent the thoughts, opinions, plans or strategies of Commvault Systems, Inc. ("Commvault") and Commvault undertakes no obligation to update, correct or modify any statements made in this forum. Any and all third party links, statements, comments, or feedback posted to, or otherwise provided by this forum, thread or post are not affiliated with, nor endorsed by, Commvault.
Commvault, Commvault and logo, the “CV” logo, Commvault Systems, Solving Forward, SIM, Singular Information Management, Simpana, Commvault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, SnapProtect, ROMS, and CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.
Close
Copyright © 2020 Commvault | All Rights Reserved. | Legal | Privacy Policy