Deduplication Engines - Information update rate

Last post 02-12-2018, 12:55 PM by deb. 11 replies.
Sort Posts: Previous Next
  • Deduplication Engines - Information update rate
    Posted: 02-07-2018, 1:38 PM

    Under Storage Resources - Deduplication Engines, there is a wealth of information relating to it.  How often does this refresh?  I still see "Data as of January 31" on most of them.  The only two that have "as of February 7" are small, and one of them is actually not in active use.

    Also most of the times are within an hour of the rest.

    Is there a periodic process that updataes this, or is there a way to cause a refresh of the displayed data?

    I'm seeing some hgh QI times on some, and would like to have the latest info.

    Thanks

  • Re: Deduplication Engines - Information update rate
    Posted: 02-07-2018, 3:51 PM
    • Ali is not online. Last active: 03-05-2018, 11:33 PM Ali
    • Top 10 Contributor
    • Joined on 08-05-2010

    Hi Deb,

    DDB stats will come in after a successful DDB backup job.

    Believe the Q&I times are on 24hr intervals, there is an "As of" column right there in the GUI which will tell you the that.

    If you want more recent, have a look at the SIDBEngine.log and look for Query and Insert times there, it will give you more real-time, that also happens on intervals every 10 or so minutes I believe.

    Another useful report is the Health Report, you need Metrics for that, presuming you have it running, unless you are using Commvault's cloud, very helpful stuff to quickly spot a bottleneck.

    http://documentation.commvault.com/commvault/v11/article?p=features/reports/metrics/health_check_view_report.htm

  • Re: Deduplication Engines - Information update rate
    Posted: 02-07-2018, 4:28 PM

    That makes sense.  I did look at the drop down on the individual DDB partition data (we have 2 partitions per) and it states that the last DDB backup was on the 31st. 

    Looking at the DDBBackup subclient on the media agent servers (we have three), and listing jobs, it is doing backups every 8 hours using the system created DDB backup schedule.  The only odd thing is that the 5pm jobs fail to start for some reason, but the ones at 8am and midnight are successful.

    Hopefully I am missing something here, but wondiring if I have an issue.

    Of the 13 or so storage policies that I have, only two show today's date, and they are the ones that really do not have any activity to note.

  • Re: Deduplication Engines - Information update rate
    Posted: 02-07-2018, 4:37 PM

    I believe I resolve the 5pm issue.  I had created a backup schedule for the default subclient on the media agent servers, and did not realize that I had include both the DDBBackup and IndexBackup in it, so it was trying to kick of at 5pm, while there was a 4pm DDBBackup that was still running.

     

  • Re: Deduplication Engines - Information update rate
    Posted: 02-07-2018, 4:59 PM
    • Ali is not online. Last active: 03-05-2018, 11:33 PM Ali
    • Top 10 Contributor
    • Joined on 08-05-2010

    Perfect, makes sense.  If you could mark this thread resolved too that would be great.

  • Re: Deduplication Engines - Information update rate
    Posted: 02-07-2018, 5:05 PM

    That did seem to resolve the 5pm issue.  But the information displayed in the "Deduplication Engines" is still out of date.

  • Re: Deduplication Engines - Information update rate
    Posted: 02-07-2018, 5:08 PM
    • Ali is not online. Last active: 03-05-2018, 11:33 PM Ali
    • Top 10 Contributor
    • Joined on 08-05-2010

    Could you share a screenshot?  What SP are you on as well?

  • Re: Deduplication Engines - Information update rate
    Posted: 02-08-2018, 5:07 AM

    Ali:

    If you want more recent, have a look at the SIDBEngine.log and look for Query and Insert times there, it will give you more real-time, that also happens on intervals every 10 or so minutes I believe.

    You can find specific time measurements by searching for these entries:

    [Query And Insert Handler], Exp Avg [0.000183], Samples [100], Total Time [59823.705828], Total Invoked [17247753], Total Avg [0.003468]
    [Commit Primary Records Handler], Exp Avg [11.017586], Samples [100], Total Time [6433.396739], Total Invoked [943], Total Avg [6.822266]
    [Prune Archive Files Handler], Exp Avg [0.000270], Samples [100], Total Time [0.463206], Total Invoked [1645], Total Avg [0.000282]
    [Prune Records Handler], Exp Avg [0.068005], Samples [100], Total Time [2769.086850], Total Invoked [22178], Total Avg [0.124857]
    [Get Prunable Records Handler], Exp Avg [0.062241], Samples [100], Total Time [4180.313688], Total Invoked [22213], Total Avg [0.188192]
    [CLSI Signture Query Handler], Exp Avg [0.002699], Samples [100], Total Time [13611.549772], Total Invoked [3563602], Total Avg [0.003820]
    [CLSI Commit Records request Handler], Exp Avg [0.000113], Samples [100], Total Time [1114.447916], Total Invoked [1280299], Total Avg [0.000870]
    [Quiesce Handler], Exp Avg [16.247366], Samples [1], Total Time [16.247366], Total Invoked [1], Total Avg [16.247366]
    [Resume Handler], Exp Avg [0.226302], Samples [1], Total Time [0.226302], Total Invoked [1], Total Avg [0.226302]
    [Sync Splits State], Exp Avg [0.000000], Samples [100], Total Time [0.000046], Total Invoked [289], Total Avg [0.000000]
    Please note that these averages are terrible regarding performance, so do not use them as a reference point.

    Jos Meijer
    Senior Technical Consultant
  • Re: Deduplication Engines - Information update rate
    Posted: 02-08-2018, 12:26 PM

    Under
        Storage Resources
            Deduplication Engines
                (Some storage policy)
                    SIDB_xxxx
                        partition 1
                        partition 2

    On the screen when selecting "Some storage policy"
        Data as of January 31...
        
    On the screen when selecting "SIDB_xxxx"
        Data as of January 31...
        DDB last verfied on 01/30/2018....
        in the Partitions section, there are two rows, each show "Last DDB Backup Time 01/31/18 18:08:00)
        in the DDB Statistics(Across Partions) section, the tables list "As of", and the last entry is 01/31/18 12:00 AM
        It lists average QI time in last 3 days as 990us

    On the screen for the individual partitions, page header of "DDB Partition - ...."
        Data as of February 08, 2018 08:37
        In properties, still shows last DDB Backup Time of 01/31/2018 08:00 AM
        in the Partition Statistics section, it lists "As of", and the entries there are current. last one shows 02/08/18 12:00 AM
        It lists average QI time in last 3 days as 927us, and 990us for the 2nd partition

    I'm still not understanding why the two previous screens show data "as of January 31", while drilling down to the individual partion screens show current.

    I am not that interested in being super current, just wanting to track some numbers ( mainly QI ) over time, and would like to understand why the different views are not sync'd up and showing statistics for the same time period.

    My main concern is the "last DDB Backup time".  Since all of the screens, even the one that seems up to date, show that the the last DDB backup is from last month?

    Is this just one of those things, or do I have a problem with my DDB Backups somewhere in the system?

    I had a screenshot, but it did not want to load.




  • Re: Deduplication Engines - Information update rate
    Posted: 02-08-2018, 5:36 PM

    Well, I've come to the conclusion that I cannot rely on the statistics with Deduplication Engines to provide any kind of current date. 

    One them has a "Storage Policy Copy - ...." Data as of January 31

    The SIDB also has, "As of January 31"

    One of the partitions has "As of February 7 1:18pm"

    The other has "As of February 8 2:25pm"

    On another, one partition is "as of Jan 31" and the other is "as of Feb 8".

    Hopefully I have no real DDB issues, it is simply a matter of inconsistancies in the data/statistics shown in the GUI.

     

  • Re: Deduplication Engines - Information update rate
    Posted: 02-12-2018, 12:36 PM

    Interesting.  I exited the commserve GUI and logged back in, and now the listings are all current.  Seems it does not refresh accurately over time.

  • Re: Deduplication Engines - Information update rate
    Posted: 02-12-2018, 12:55 PM

    Having just said that, some of the data still seems our of date.  Primarily the "Last DDB Backup Time" is still several days old on some of them.

    Some have a last time of 2/9, and some 2/12

    Exited the GUI again (last exit was on the 9th), and now they are up to date.

    Definately does not refresh well.

    I'll mark this as resolved, even though it is still annoying.

The content of the forums, threads and posts reflects the thoughts and opinions of each author, and does not represent the thoughts, opinions, plans or strategies of Commvault Systems, Inc. ("Commvault") and Commvault undertakes no obligation to update, correct or modify any statements made in this forum. Any and all third party links, statements, comments, or feedback posted to, or otherwise provided by this forum, thread or post are not affiliated with, nor endorsed by, Commvault.
Commvault, Commvault and logo, the “CV” logo, Commvault Systems, Solving Forward, SIM, Singular Information Management, Simpana, Commvault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, SnapProtect, ROMS, and CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.
Close
Copyright © 2018 Commvault | All Rights Reserved. | Legal | Privacy Policy