Measure Disk Performance

Last post 01-13-2011, 12:40 PM by Stephen Connington. 11 replies.
Sort Posts: Previous Next
  • Measure Disk Performance
    Posted: 10-29-2010, 12:59 PM
    • VC0 is not online. Last active: 06-10-2011, 4:31 PM VC0
    • Top 500 Contributor
    • Joined on 06-22-2010
    • Texas
    • Novice
    • Points 37

    I have carved out a lun for dedupe to check and see if it meets the requirements.

    We are going to start deduping in a couple of months and want to see if we have a good foundation.

    But I can't find the SIDB Simulator executable to run on a windows media agent.

    We have not purchased the CLA yet but need to know if our older san EMC CX500 10k FC Drives will handle dedup.

    In less than a year we will be upgrading our new CX4-120 that is full to a CX4-240, but untill then I am going to haft to use the CX500 that only have 2 gridstor X64 media agents on it.

    Also I am looking for best practices for dedup, the lun that was created as raid 1_0 with 10 fc drives across enclosers other than that I used defaults.

  • Re: Measure Disk Performance
    Posted: 10-29-2010, 2:11 PM

    simtool should be part of the resource pack which you can download here:

    https://services.commvault.com/mainadv/resource_pack.asp?fv=8.0

  • Re: Measure Disk Performance
    Posted: 11-01-2010, 4:43 PM
    • VC0 is not online. Last active: 06-10-2011, 4:31 PM VC0
    • Top 500 Contributor
    • Joined on 06-22-2010
    • Texas
    • Novice
    • Points 37

    Can I get the exact file name? I have looked everywhere for the executible for windows X64.

    Do you need to own the license for dedup to get the simtool?

     

  • Re: Measure Disk Performance
    Posted: 11-01-2010, 5:13 PM

    Here is the ReadMe info:

    from \ResourcePack\8.0.0\ForCDAndWeb\INTEL32\Windows\Media Management\SIDBSimulationTool

    SIDB Simulator

    Scope

    Intended Audience
    All X
    CommVault Internal  
    CE/System Test  
    Development  

    Release

    8.0.0

    Description

    This tool is used to estimate the performance of the disk where you plan to create the deduplication store. You can use this tool to determine whether the disk, where you are planning to create a deduplication store, is performing optimally. This tool simulates the deduplication operation by querying and inserting  a large number of records into the Deduplication Database. During simulation, when the program detects slowness in the system, informational events are printed out .

    Use this tool to get the following estimates about the performance of the disk:

    • The Query & Insert average time for a given application data size.
    • The maximum size of application data that can be backed up using the deduplication database for a given time limit.

    Usage

    Running the Utility
    1. Copy the SIDB folder from the Resource Pack to the disk where you want to create the Deduplication Store.
    2. Run the sidb_sim.jar. The SIDB Simulator dialog box displays the following options:
      • Determine max data handled by the dedup database for given threshold query response time
      • Determine the average query response time for given application data size

     

    Determining the maximum application data size 

    You can find out the maximum size of the application data that can be backed up using the deduplication database for a given access time limit for each record.

    To find out the maximum size of the application data:

    1. On the SIDB Simulator dialog box, select Determine max data handled by the dedup database for given threshold query response time.
    2. In the Directory to use for Simulation, specify the directory and click Continue.
    3. Specify the following parameters:
      • Time Limit  - The access time limit for each record in microsecond.
      • Dedup Ratio - The expected deduplicaiton ratio
      • Block Size - The deduplication data block size in KB
      • Iterations for Sampling Query Time - Number of iterations performed to determine the query and insert time spent at a particular application datasize. The default value is 10000 iterations.
      • Continue Simulation upto - The percentage of the threshold time limit upto which the Simulation can continue after the threshold has reached. For example, when you specify 10000 microseconds as the threshold time limit and set the Continue Simulation upto  value to 50%, the simulation will continue upto 15000 microseconds. 
    4. If you want to estimate the disk performance statistics, select the Measure Disk Performance check box.
    5. If you want to keep the simulation data after the simulation is completed, select the keep db files after simulation check box.

      Note

      If you do not select this option, all the files in the simulation directory will be deleted after the simulation is complete.

    6. Click Next.
    7. The SIDB Simulator dialog box displays the parameters and its values. The tool will use these parameters to run the simulation. Click Start to initiate the simulation process.
    8. The Simulation Report tab of the SIDB Simulator dialog box displays the progress of the simulation process. Also a graph of Query Time against Data size is displayed. Click Maximize to see the details in the graph.

      Note

      Each data point in the graph is calculated from the average of the last x iterations; where x is the number of Iterations for Sampling Query.

    1. The Results tab of the SIDB Simulator dialog box displays the results of the simulation. The results display the maximum size of the application data that can be backed up using the deduplication database for the given access time limit.

    You can also view the results of the simulation in the cmd_out.log file. This file will be available in the SIDB folder on your disk.

     

    Determining the average query response time

    You can find out the the average query response time for each record for a given data size.

    To find out the average query response time:

    1. On the SIDB Simulator dialog box, select Determine the average query response time for given application data size.
    2. In the Directory to use for Simulation, specify the directory and click Continue.
    3. Specify the following parameters:
      • Data Size - the application data size in GB
      • Dedup Ratio - The expected deduplicaiton ratio
      • Block Size - The deduplication data block size in KB
      • Iterations for Sampling Query Time - Number of iterations performed to determine the query and insert time spent at a particular application datasize. The default value is 10000 iterations. 
      • Continue Simulation upto - The percentage of the threshold time limit upto which the Simulation can continue after the threshold has reached. For example, when you specify 10 GB as the threshold data size and set the Continue Simulation upto  value to 50%, the simulation will continue upto 15GB.
    4. If you want to estimate the disk performance statistics, select the Measure Disk Performance check box.
    5. If you want to keep the simulation data after the simulation is completed, select the keep db files after simulation check box.

      Note

      If you do not select this option, all the files in the simulation directory will be deleted after the simulation is complete.

    6. Click Next.
    7. The SIDB Simulator dialog box displays the parameters and its values. The tool will use these parameters to run the simulation. Click Start to initiate the simulation process.
    8. The Simulation Report tab of the SIDB Simulator dialog box displays the progress of the simulation process. Also a graph of Query Time against Data size is displayed. Click Maximize to see the details in the graph.

      Note

      Each data point in the graph is calculated from the average of the last n iterations; where n is the number of Iterations for Sampling Query.

    9. The Results tab of the SIDB Simulator dialog box displays the results of the simulation. The results contain the estimated average transaction time for an insert/query in the deduplication database based on the size of the application data that is backed up.

    You can also view the results of the simulation in the cmd_out.log file. This file will be available in the SIDB folder on your disk.

  • Re: Measure Disk Performance
    Posted: 11-03-2010, 10:38 AM

    VC0,

    Thanks for the post.

    The CS500 should be fine for the DDB. CV recommends having the DDB on a Raid0 because its the fast I/O available. But most companies are not comfortable with that so Raid 1+0 is fine. the DDB should have its own spindles with a single lun. This is to prevent splitting the I/O between other partitions and limiting ddb growth. The other important configuration is the MA its self.

    Cv recommend having the O/S on a Raid 1 and more importantly the Page file need to have its own Raid 0 spindles.  the memory should be 12gb min. for 8 and 24gb min. for 9.

     

    If runing Windows 2008 R2 make sure to have the following MS hotfix

    http://support.microsoft.com/default.aspx?scid=kb;EN-US;982383

    If the CX500 is going to be used for the disk library as well you will want to use seperate interfaces if possible.

    the sim tool is part of the SIDB2 executable found in the base directory of the Media Agent.

    This tool is a command line tool that will show how well the DDB will scale based on the drive configuration. When running this tool make sure to run it concurrently the same number of time there will be active DDB running during production. this is to simulate real world load. CV does not recommend running more than 4 ddbs per MA especially on the same DDB lun. typically we find that a properly configured MA can run the two Sim tool to completion. 

    how much data are you looking to dedup? how many storage policies are you configuring with dedup?

    let me know if you have any questions.

    AaronA

     

  • Re: Measure Disk Performance
    Posted: 11-04-2010, 10:15 AM
    • VC0 is not online. Last active: 06-10-2011, 4:31 PM VC0
    • Top 500 Contributor
    • Joined on 06-22-2010
    • Texas
    • Novice
    • Points 37

    Thanks All

    About 12 Tb on weekly full, and 7-8 storage policies.

    What does not dedup well, we have windows filesystem, sql, sharepoint, exchange, ad, centos and image server and vmware.

  • Re: Measure Disk Performance
    Posted: 11-10-2010, 11:57 AM

    VC0,

    if you are not getting a good dedup ratio then i would recommend opening a ticket to have this looked at.

    You can also verify that Dedupilcation it enabled on each subclient. This is a huge source of deduplication ratio issue. the easiest way to determine if dedup is turn on at the client is to run a Jobs in Storage Policy report only on fulls for the last 30 days. Compare the size of backup to the size on disk. if they numbers are always close in size to each other than most likely dedup is not turned on.  

    AaronA

  • Re: Measure Disk Performance
    Posted: 11-10-2010, 1:12 PM
    • VC0 is not online. Last active: 06-10-2011, 4:31 PM VC0
    • Top 500 Contributor
    • Joined on 06-22-2010
    • Texas
    • Novice
    • Points 37

    AaronA

    We don't have dedup yet, I was trying to prepare for the CLA that we will be getting soon. at that point dedupe and archive will be implemented. and wanted to check the performance of our older CX500 san and the best way to setup the raid groups and luns like raid 1+0, we will blow everything on san away and totaly reconfigure for CV only. I am trying to get a good foundation so we don't have problems when the se comes to install. Also need to look at the media agents. Thanks for the info we need all we can get.

  • Re: Measure Disk Performance
    Posted: 11-11-2010, 3:56 PM

    VCO

     

    Good luck with dedup deployment!

    AaronA

  • Re: Measure Disk Performance
    Posted: 11-11-2010, 4:35 PM
    • VC0 is not online. Last active: 06-10-2011, 4:31 PM VC0
    • Top 500 Contributor
    • Joined on 06-22-2010
    • Texas
    • Novice
    • Points 37
    I am not sure how to take that!
  • Re: Measure Disk Performance
    Posted: 01-13-2011, 12:27 PM

    Looking through the resource pack currently available it doesn't appear to me that the SIDB Simulator is included anymore...

    https://services.commvault.com/mainadv/resource_pack.asp?fv=8.0

    -Chad

  • Re: Measure Disk Performance
    Posted: 01-13-2011, 12:40 PM

    Hi Chad,

    That file was there.... I will look into where the link dissapeared too.

    In the meantime I have attached the utility - same instructions above for use.

The content of the forums, threads and posts reflects the thoughts and opinions of each author, and does not represent the thoughts, opinions, plans or strategies of Commvault Systems, Inc. ("Commvault") and Commvault undertakes no obligation to update, correct or modify any statements made in this forum. Any and all third party links, statements, comments, or feedback posted to, or otherwise provided by this forum, thread or post are not affiliated with, nor endorsed by, Commvault.
Commvault, Commvault and logo, the “CV” logo, Commvault Systems, Solving Forward, SIM, Singular Information Management, Simpana, Commvault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, SnapProtect, ROMS, and CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.
Close
Copyright © 2019 Commvault | All Rights Reserved. | Legal | Privacy Policy