[Bug]: TF400324: Azure DevOps services are not available from server https://dev.azure.com/org

Question

[Bug]: TF400324: Azure DevOps services are not available from server https://dev.azure.com/org

AjinkyaIG opened this issue 9 months ago · 1 comments

AjinkyaIG commented 9 months ago

Version

I confirm that I am using the latest version

Source Version

Azure DevOps Service

Target Version

Azure DevOps Service

Relevant configuration

{
    "ChangeSetMappingFile": null,
    "Source": {
      "$type": "TfsTeamProjectConfig",
      "Collection": "https://dev.azure.com/source/",
      "Project": "abc",
      "ReflectedWorkItemIDFieldName": "Custom.ReflectedWorkItemId",
      "AllowCrossProjectLinking": false,
      "AuthenticationMode": "AccessToken",
      "PersonalAccessToken": "",
      "PersonalAccessTokenVariableName": "",
      "LanguageMaps": {
        "AreaPath": "Area",
        "IterationPath": "Iteration"
      }
    },
    "Target": {
      "$type": "TfsTeamProjectConfig",
      "Collection": "https://dev.azure.com/target/",
      "Project": "xyz",
      "ReflectedWorkItemIDFieldName": "Custom.ReflectedWorkItemId",
      "AllowCrossProjectLinking": false,
      "AuthenticationMode": "AccessToken",
      "PersonalAccessToken": "",
      "PersonalAccessTokenVariableName": "",
      "LanguageMaps": {
        "AreaPath": "Area",
        "IterationPath": "Iteration"
      }
    }, 
    "FieldMaps": [],
    "GitRepoMapping": null,
    "LogLevel": "Information",
    "CommonEnrichersConfig": null,
    "Processors": [
      {
        "$type": "WorkItemMigrationConfig",
        "Enabled": true,
        "ReplayRevisions": true,
        "PrefixProjectToNodes": false,
        "UpdateCreatedDate": true,
        "UpdateCreatedBy": true,
        "WIQLQueryBit": "AND [System.WorkItemType] NOT IN ('Shared Steps')",
        "WIQLOrderBit": "[System.ChangedDate] desc",
        "LinkMigration": true,
        "AttachmentMigration": true,
        "AttachmentWorkingPath": "c:\\temp\\WorkItemAttachmentWorkingFolder\\",
        "FixHtmlAttachmentLinks": false,
        "SkipToFinalRevisedWorkItemType": false,
        "WorkItemCreateRetryLimit": 5,
        "FilterWorkItemsThatAlreadyExistInTarget": true,
        "PauseAfterEachWorkItem": false,
        "AttachmentMaxSize": 480000000,
        "AttachRevisionHistory": true,
        "LinkMigrationSaveEachAsAdded": false,
        "GenerateMigrationComment": false,
        "WorkItemIDs": null,
        "MaxRevisions": 0,
        "NodeStructureEnricherEnabled": null,
        "UseCommonNodeStructureEnricherConfig": false,
        "NodeBasePaths": null,
        "AreaMaps": {},
        "IterationMaps": {},
        "MaxGracefulFailures": 0,
        "SkipRevisionWithInvalidIterationPath": false
      }
    ],
    "Version": "14.3",
    "workaroundForQuerySOAPBugEnabled": false,
    "WorkItemTypeDefinition": {
      "sourceWorkItemTypeName": "targetWorkItemTypeName"
    },
    "Endpoints": {}
  }

Relevant log output

2024-01-13 11:08:05.967 +05:30 [INF] 8631 Work items received, verifying
2024-01-13 11:08:31.005 +05:30 [INF] Loading 8631 Work Items
2024-01-13 11:09:01.089 +05:30 [INF] 138/8631 1.6%
2024-01-13 11:09:31.134 +05:30 [INF] 312/8631 3.61%
2024-01-13 11:10:01.208 +05:30 [INF] DONE in 00:01:55.5337536
2024-01-13 11:10:01.265 +05:30 [FTL] Error while running WorkItemMigration
Microsoft.TeamFoundation.TeamFoundationServiceUnavailableException: TF400324: Azure DevOps services are not available from server https://dev.azure.com/target/.
Technical information (for administrator):
  The underlying connection was closed: A connection that was expected to be kept alive was closed by the server. ---> System.Net.WebException: The underlying connection was closed: A connection that was expected to be kept alive was closed by the server. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count)
   at System.Net.Security._SslStream.StartFrameHeader(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security._SslStream.StartReading(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security._SslStream.ProcessRead(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.TlsStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead)
   --- End of inner exception stack trace ---
   at System.Net.HttpWebRequest.GetResponse()
   at Microsoft.TeamFoundation.Client.Channels.TfsHttpWebRequest.SendRequestAndGetResponse(HttpWebRequest webRequest, WebException& webException)
   --- End of inner exception stack trace ---
   at Microsoft.TeamFoundation.Client.Channels.TfsHttpWebRequest.ReadResponse(HttpWebResponse webResponse, WebException webException)
   at Microsoft.TeamFoundation.Client.Channels.TfsHttpWebRequest.SendRequest()
   at Microsoft.TeamFoundation.Client.Channels.TfsHttpRequestChannel.Request(TfsMessage message, TimeSpan timeout)
   at Microsoft.TeamFoundation.Client.Channels.TfsHttpClientBase.Invoke(TfsClientOperation operation, Object[] parameters, TimeSpan timeout, Object[]& outputs)
   at Microsoft.TeamFoundation.WorkItemTracking.Proxy.ClientService.GetWorkItem(Int32 workItemId, Int32 revisionId, Int32 minimumRevisionId, Nullable`1 asOfDate, Boolean useMaster, RowSetCollection& workItem, MetadataTableHaveEntry[] metadataHave, String& dbStamp, RowSetCollection& metadata)
   at Microsoft.TeamFoundation.WorkItemTracking.Proxy.WorkItemServer.GetWorkItem(String requestId, Int32 workItemId, Int32 revisionId, Int32 minimumRevisionId, Nullable`1 asOfDate, Boolean useMaster, IWorkItemRowSets& workItem, MetadataTableHaveEntry[] metadataHave, String& dbStamp, IMetadataRowSets& metadata)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore.LoadWorkItem(Int32 id, Int32 rev, Nullable`1 asof, IWorkItemRowSets& witem)
   at Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItem.EnsureCollections()
   at Microsoft.TeamFoundation.WorkItemTracking.Client.LinkCollection.Populate()
   at Microsoft.TeamFoundation.WorkItemTracking.Client.LinkCollection.get_Count()
   at Microsoft.TeamFoundation.WorkItemTracking.Common.Enumerator`1.MoveNext()
   at MigrationTools.Endpoints.TfsWorkItemConvertor.GetLinkItems(LinkCollection tfsLinks) in D:\a\1\s\src\MigrationTools.Clients.AzureDevops.ObjectModel\Endpoints\TfsWorkItemConvertor.cs:line 92
   at MigrationTools.Endpoints.TfsWorkItemConvertor.MapWorkItemtoWorkItemData(WorkItemData context_wid, WorkItem context_wi, Dictionary`2 fieldsOfRevision) in D:\a\1\s\src\MigrationTools.Clients.AzureDevops.ObjectModel\Endpoints\TfsWorkItemConvertor.cs:line 37
   at MigrationTools.TfsExtensions.ToWorkItemDataList(IList`1 collection) in D:\a\1\s\src\MigrationTools.Clients.AzureDevops.ObjectModel\TfsExtensions.cs:line 163
   at VstsSyncMigrator.Engine.WorkItemMigrationContext.InternalExecute() in D:\a\1\s\src\VstsSyncMigrator.Core\Execution\MigrationContext\WorkItemMigrationContext.cs:line 171
   at MigrationTools._EngineV1.Processors.MigrationProcessorBase.Execute() in D:\a\1\s\src\MigrationTools\_EngineV1\Processors\MigrationProcessorBase.cs:line 47
2024-01-13 11:10:01.282 +05:30 [ERR] WorkItemMigration The Processor MigrationEngine entered the failed state...stopping run

What happened?

I was in the middle of a migration, out of 8631, around 1300 items were migrated already.
And then the process stopped with above error.
First it was for the source ADO org- TF400324: Azure DevOps services are not available from server https://dev.azure.com/source/

Then I tried re-running the migration, then it shows the same error for the target ADO org.

Why is this happening? Have I reached any limit in terms of migration?

Debug in Visual Studio

Visual Studio Debug

Answer 1 · 2024-01-15T09:53:14.000Z

No idea @AjinkyaIG you might need to check with Microsoft...

You can check if you are being rate limited on https://dev.azure.com/org/_settings/usage but you should not get a "service unavailable".