New media agents can't connect to themselves?

Last post 05-15-2019, 3:17 AM by Wwong. 6 replies.
Sort Posts: Previous Next
  • New media agents can't connect to themselves?
    Posted: 05-09-2019, 7:49 PM

    Hi,

    I'm setting up some new media agents, and they're working for everything except cloud copies, which is when I get this weirdness:

    Error Code: [13:138]
    Description: Error occurred while processing chunk [46033113] in media [V_5795395], at the time of error in library [.Windows_LocalDisk] and mount path [[andormu] J:\PsuBackups], for storage policy [.Windows] copy [S3] MediaAgent [andormu]: Backup Job [3254614]. Unable to setup the copy pipeline. Please check connectivity between Source MA [andormu] and Destination MA [andormu].
    Source: andormu, Process: CVJobReplicatorODS

    Error occurred while processing chunk [46033105] in media [V_5795514], at the time of error in library [.Windows_LocalDisk] and mount path [[nozari] J:\PsuBackups], for storage policy [.Windows] copy [S3] MediaAgent [nozari]: Backup Job [3254610]. Unable to setup the copy pipeline. Please check connectivity between Source MA [nozari] and Destination MA [nozari].
    Source: nozari, Process: CVJobReplicatorODS

    I've tried searching the internet and this forum, but (as expected) all I can find related to this error is actual network connectivity issues, not this "can't-connect-to-myself" nonsense.

    Any ideas?

  • Re: New media agents can't connect to themselves?
    Posted: 05-09-2019, 8:33 PM

    Hi Sheree

    Can you please confirm with the new MediaAgent, is this a Local MediaAgent or a Cloud MediaAgent?

    Also while running Aux Copy to the Cloud Library, is this going through the new MediaAgent to the Cloud MediaAgent or directly writing to the Cloud Library ?

    At the moment I would recommend using the CloudTestTool to just test the general connectivity between MediaAgent and Cloud, and see if there are any latency that could result in "Unable to setup the copy pipeline"

    In most case when Commvault reports "Unable to setup the copy pipeline", this is an indication that the initial pipeline established between the Commvault MediaAgent and the Cloud Library for write process has been interrupted, which could be a result of various factor (i.e. environment, process crash, timeouts, latency and etc). 

    But to narrow it further it is recommended to rule out factors such as environmental (connectivity), by testing the upload/download speed between MediaAgent and Cloud 

    Link - http://documentation.commvault.com/commvault/v11/article?p=9228.htm

    If you can give the above a go and share the results that would be great.

    Regards

    Winston 

     

  • Re: New media agents can't connect to themselves?
    Posted: 05-09-2019, 9:59 PM

    Hi Winston,

     

    It looks like that specific problem went away when I re-created the mount paths as 'Amazon S3' instead of 'S3 Compatible'... So I guess this means S3 isn't S3-compatible? Tongue out

    For the record, CloudTestTool reported:
    File size = 4096.00 MB is uploaded in [72] seconds, transfer rate 56.89 MByte/sec.

    Here's the new errror:

    Error Code: [13:138]
    Description: Error occurred while processing chunk [46033105] in media [V_5795514], at the time of error in library [.Windows_LocalDisk] and mount path [[nozari] J:\PsuBackups], for storage policy [.Windows] copy [S3] MediaAgent [chronormu]: Backup Job [3254610]. Cannot impersonate user. User credentials provided for disk mount path access may be incorrect.
    Source: chronormu, Process: CVJobReplicatorODS

    Either one Media Agent is trying to access the other Media Agent's local disk (which it obviously can't) or the error descriptions are wonky.  This credential does definitely work for the shared mount paths, since that library's used by other jobs that work fine.

  • Re: New media agents can't connect to themselves?
    Posted: 05-11-2019, 12:56 PM
    • Aplynx is not online. Last active: 05-13-2019, 12:56 PM Liam
    • Top 10 Contributor
    • Joined on 05-04-2010
    • New Jersey
    • Master
    • Points 1,581

    The cvma.log should show the error and why. 

  • Re: New media agents can't connect to themselves?
    Posted: 05-13-2019, 8:13 AM

    Hi Sheree

    S3 Compatible, refers to other Vendors that is supported by CommVault using the S3 API calls - https://documentation.commvault.com/commvault/v11_sp14/article?p=9237.htm

    For the other issue, where you are not getting Process CHUNK errors, can you please confirm for your Local MediaAgent which is connected to the local Disk Library, is this also shared to other MediaAgent via CommVault (using Data Server IP)?

    From the errors, the Mount Path looks to be local (LUN or iSCSI base) and then provisioned as NTFS from Windows (J:\)

    Can you please share the CVJobReplicatorODS.log & CVMA.log at the time of the issue when Aux Copy is running? This will provide more details on why the Process CHUNK error is being reported

    Regards

    Winston 

  • Re: New media agents can't connect to themselves?
    Posted: 05-14-2019, 3:24 PM

    Hi Winston,

    I'm curious about that whole 'Data Server IP' thing, because removing the auto-created 'Data Server IP' sharing configuration and manually sharing as SMB is what fixed this issue earlier for the primary/local copy.  What's a 'Data Server IP' connection?

    The same credential is being used elsewhere by these media agents for putting data on each other's local disks via SMB, so this is definitely not a 'bad credential' problem.

    This appears to be fixed--The credential account needed write access to the Commvault log files.

    Thanks!

    Sheree

  • Re: New media agents can't connect to themselves?
    Posted: 05-15-2019, 3:17 AM

    Hi Sheree

    Good to know you were able to sort out the issue with sharing of Mount path

    In relation to "Data Server IP" this is a Commvault feature that utilizes NFS protocol to share local volume to peer MediaAgent. This reduces the need to configure CIF share manually (entering the credentials one at a time)

    When using Data Server IP, when a volume on MA1 is required by MA2, MA2 will send an export request to MA1 to export the required Volume using NFS (at the application layer). It will then select a high range Dynamic Port to facilitate the mount request.

    The feature iteslef is to mitigate from need to configure stuff outside of Commvault. However if the export/mount request is not successful, it will result in the errors that you saw in your other Aux Copy Job. 

    There are logs such as 3dnfs.log and NTFSTransport.log on the MediaAgent that facilitates the export/mount request and this shoudl shed more light on the reason of the failure. 

    Hopefully that helps 

    Regards

    Winston 

The content of the forums, threads and posts reflects the thoughts and opinions of each author, and does not represent the thoughts, opinions, plans or strategies of Commvault Systems, Inc. ("Commvault") and Commvault undertakes no obligation to update, correct or modify any statements made in this forum. Any and all third party links, statements, comments, or feedback posted to, or otherwise provided by this forum, thread or post are not affiliated with, nor endorsed by, Commvault.
Commvault, Commvault and logo, the “CV” logo, Commvault Systems, Solving Forward, SIM, Singular Information Management, Simpana, Commvault Galaxy, Unified Data Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault, QuickSnap, QSnap, Recovery Director, CommServe, CommCell, SnapProtect, ROMS, and CommValue, are trademarks or registered trademarks of Commvault Systems, Inc. All other third party brands, products, service names, trademarks, or registered service marks are the property of and used to identify the products or services of their respective owners. All specifications are subject to change without notice.
Close
Copyright © 2019 Commvault | All Rights Reserved. | Legal | Privacy Policy