Load balanced pruning on paritioned DDB

Last post 02-13-2017, 1:41 PM by Ali. 2 replies.
Sort Posts: Previous Next
  • Load balanced pruning on paritioned DDB
    Posted: 02-09-2017, 12:34 AM

    Request is:  Commserve should either round robin pruning requests or even better would be if each MA worked down it's own DDB's pending deletes only (checkbox option?).  The commserve currently seems to have a preferred MA and uses another only if the primary is busy.

    Background:

    Searched the forums and found this prior post that's asking for the same thing.  https://forum.commvault.com/forums/thread/46007.aspx

     

    There seems to be an existing checkbox that's headed in the right direction:

    http://documentation.commvault.com/commvault/v11/article?p=features/disk_library/online_help_mp.htm#Mount_Path_Properties__Allocation_Policy_

    "Select Mediaagents for pruning aged data based on the associated copies datapath"

    However in this case it'd be select mediaagent for pruning based on wether we're processing zeroref/pending deletes from that MA's DDB partition or not.

    This would also bypass the "error=0x80070013:{IndexHdrMagic>::LockFile(753)/W32.19.(Error=19)-Unable to aquire the Lock to File " errors that we get when MA's try to prune the same file.  Since each prune would be unique to the MA with that record in it's DDB partition.

     

    One of our DDB partitions is 1.2pb app size accross a 4 MA wide partitioned DDB.  All maglibs are shared between the mediaagents via network storage (NFS/CIFS).  Cifs is due to hole drilling only..

    Currently 1 MA is doing 80% of the pruning, another is doing 20% and the other 2 are sitting idle.

    I know that each MA has it's own list of pending delete (zerorefs) to process.  Currently we've got around 60 million on each MA including those that are not currently pruning at all.

    As discussed on case: 170205-5 it looks like the commserve assigns out the pruning to a particular MA ie. the below mediamanager.log:

     

    MA X does 80%, Y does 20% and Z1/Z2 sit there doing nothing and don't even appear in the logs.

    mediamanager.log:
    10436 1bec  02/08 23:10:10 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [30, Y],RID [1022209/0/0]

    10436 1bec  02/08 23:10:10 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [33, X],RID [1022210/0/0]
    10436 5218  02/09 00:15:10 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [30, Y],RID [1028641/0/0]
    10436 5218  02/09 00:15:10 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [33, X],RID [1028642/0/0]
    10436 7168  02/09 01:15:11 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [33, X],RID [1032110/0/0]
    10436 9428  02/09 02:15:14 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [33, X],RID [1033651/0/0]
    10436 91d8  02/09 03:20:10 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [33, X],RID [1035146/0/0]
    10436 9254  02/09 04:20:09 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [33, X],RID [1036576/0/0]
    10436 9428  02/09 05:20:10 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [33, X],RID [1037933/0/0]
    10436 2724  02/09 06:25:10 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [33, X],RID [1039551/0/0]
    10436 45ac  02/09 07:25:37 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [30, Y],RID [1040933/0/0]
    10436 45ac  02/09 07:25:37 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [33, X],RID [1040934/0/0]
    10436 25b8  02/09 08:25:15 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [30, Y],RID [1042360/0/0]
    10436 25b8  02/09 08:25:15 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [33, X],RID [1042361/0/0]
    10436 45ac  02/09 09:30:12 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [30, Y],RID [1044705/0/0]
    10436 45ac  02/09 09:30:12 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [33, X],RID [1044706/0/0]
    10436 9254  02/09 10:30:10 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [33, X],RID [1049204/0/0]
    10436 2724  02/09 11:30:11 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [30, Y],RID [1057652/0/0]
    10436 2724  02/09 11:30:11 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [33, X],RID [1057653/0/0]
    10436 5218  02/09 12:30:17 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [30, Y],RID [1064727/0/0]
    10436 5218  02/09 12:30:17 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [33, X],RID [1064728/0/0]
    10436 5e04  02/09 13:35:11 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [30, Y],RID [1074571/0/0]
    10436 5e04  02/09 13:35:11 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [33, X],RID [1074572/0/0]
    10436 2724  02/09 14:35:21 -----  SERVICE  [      ]  PRUNE SIDB DATA for SIDB [13] on host [30, Y],RID [1084647/0/0]

     



  • Re: Load balanced pruning on paritioned DDB
    Posted: 02-10-2017, 1:35 AM

    good piece of information


    Vitta Bikshapathi
  • Re: Load balanced pruning on paritioned DDB
    Posted: 02-13-2017, 1:41 PM
    • Ali is not online. Last active: 07-03-2019, 12:32 PM Ali
    • Top 10 Contributor
    • Joined on 08-05-2010

    Thanks for the feedback, tracking this one for you as well.

The content of the forums, threads and posts reflects the thoughts and opinions of each author, and does not represent the thoughts, opinions, plans or strategies of Commvault Systems, Inc. ("Commvault") and Commvault undertakes no obligation to update, correct or modify any statements made in this forum. Any and all third party links, statements, comments, or feedback posted to, or otherwise provided by this forum, thread or post are not affiliated with, nor endorsed by, Commvault.
Commvault, Commvault and logo, the “CV” logo, Commvault Systems, Solving Forward, SIM, Singular Information Management, Simpana, Commvault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, SnapProtect, ROMS, and CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.
Close
Copyright © 2019 Commvault | All Rights Reserved. | Legal | Privacy Policy