Can this plugin manage manually created VMs?

Question

Can this plugin manage manually created VMs?

doxxx opened this issue 3 years ago · 3 comments

Description

The documentation says that it's possible for this plugin to manage existing manually created VMs, but I am unable to figure how to do so. I've created the cloud profile but there's no indication that it's managing my already existing agents that are running on Azure VMs. When I try clicking the Add Image button in the cloud profile, I only see options for creating new VMs from images of various kinds.

Environment

TeamCity version: 2021.2.3 (build 99711)
Azure plugin version: 0.9.9

The agent VMs are manually provisioned using Ansible. In their buildAgent.properties file, I've set the azure.instance.name to the full name of the VM in Azure.

Diagnostic logs

I see the following in the agent log:

[2022-03-31 20:34:01,411]   INFO - SpotInstanceTerminationChecker - Subscribe on Azure scheduledEvents metadata
[2022-03-31 20:34:01,413]   INFO - ver.clouds.azure.FileUtilsImpl - Reading file /var/lib/waagent/ovf-env.xml contents with sudo
[2022-03-31 20:34:01,491]   WARN - uds.azure.UnixCustomDataReader - CustomData element in file /var/lib/waagent/ovf-env.xml is empty

There's nothing interesting in the teamcity-clouds.log on the server, just the following over and over:

[2022-03-31 21:04:55,868]   INFO [ueue executor 1] - tler.AzureThrottlerAdapterImpl - [1-ReadAdapter] info: Default reads: 12000, Remaining reads: 11997, Window start time: 2022-03-31T17:36:17.194517700, Window width: PT0S, Throttler time: PT0.3S
[2022-03-31 21:04:55,868]   INFO [ueue executor 1] - tler.AzureThrottlerAdapterImpl - [7-ReadAdapter] info: Default reads: 12000, Remaining reads: 11911, Window start time: 2022-03-31T20:21:38.420460300, Window width: PT16M42.552S, Throttler time: PT0.3S
[2022-03-31 21:04:59,760]   INFO [ueue executor 1] - m.throttler.AzureThrottlerImpl - Tasks statistics: Now: 2022-03-31T21:04:59.760215900; Flow: Normal; Task: DeleteResourceGroup, Last updated: -999999999-01-01T00:00, Cache timeout: 60, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: StopVirtualMachine, Last updated: -999999999-01-01T00:00, Cache timeout: 60, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: CreateDeployment, Last updated: -999999999-01-01T00:00, Cache timeout: 60, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: DeleteDeployment, Last updated: -999999999-01-01T00:00, Cache timeout: 60, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: CreateResourceGroup, Last updated: -999999999-01-01T00:00, Cache timeout: 60, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: StartVirtualMachine, Last updated: -999999999-01-01T00:00, Cache timeout: 60, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: RestartVirtualMachine, Last updated: -999999999-01-01T00:00, Cache timeout: 60, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;
[2022-03-31 21:04:59,760]   INFO [ueue executor 1] - tler.AzureThrottlerAdapterImpl - [2-ActionAdapter] info: Default reads: 12000, Remaining reads: 12000, Window start time: 2022-03-31T17:36:19.803869100, Window width: PT0S, Throttler time: PT0S
[2022-03-31 21:05:05,792]   INFO [ueue executor 1] - m.throttler.AzureThrottlerImpl - Tasks statistics: Now: 2022-03-31T21:05:05.792972500; Flow: Normal; Task: DeleteResourceGroup, Last updated: -999999999-01-01T00:00, Cache timeout: 60, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: StopVirtualMachine, Last updated: -999999999-01-01T00:00, Cache timeout: 60, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: CreateDeployment, Last updated: -999999999-01-01T00:00, Cache timeout: 60, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: DeleteDeployment, Last updated: -999999999-01-01T00:00, Cache timeout: 60, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: CreateResourceGroup, Last updated: -999999999-01-01T00:00, Cache timeout: 60, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: StartVirtualMachine, Last updated: -999999999-01-01T00:00, Cache timeout: 60, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: RestartVirtualMachine, Last updated: -999999999-01-01T00:00, Cache timeout: 60, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;
[2022-03-31 21:05:05,792]   INFO [ueue executor 1] - m.throttler.AzureThrottlerImpl - Tasks statistics: Now: 2022-03-31T21:05:05.792972500; Flow: Normal; Task: FetchVirtualMachines, Last updated: -999999999-01-01T00:00, Cache timeout: 90, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: FetchCustomImages, Last updated: -999999999-01-01T00:00, Cache timeout: 150, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: FetchNetworks, Last updated: -999999999-01-01T00:00, Cache timeout: 90, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: FetchServices, Last updated: -999999999-01-01T00:00, Cache timeout: 150, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: FetchVirtualMachineSizes, Last updated: -999999999-01-01T00:00, Cache timeout: 90, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: FetchResourceGroups, Last updated: -999999999-01-01T00:00, Cache timeout: 150, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: FetchInstances, Last updated: -999999999-01-01T00:00, Cache timeout: 150, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: FetchStorageAccounts, Last updated: -999999999-01-01T00:00, Cache timeout: 90, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: FetchLocations, Last updated: -999999999-01-01T00:00, Cache timeout: 90, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;Task: FetchSubscriptions, Last updated: -999999999-01-01T00:00, Cache timeout: 90, ExecutionCallCount: null, RequestCallCount: null, ResourceRequestsCount: null;
[2022-03-31 21:05:05,792]   INFO [ueue executor 1] - tler.AzureThrottlerAdapterImpl - [6-ActionAdapter] info: Default reads: 12000, Remaining reads: 12000, Window start time: 2022-03-31T20:03:15.502303700, Window width: PT0S, Throttler time: PT0S
[2022-03-31 21:05:05,792]   INFO [ueue executor 1] - tler.AzureThrottlerAdapterImpl - [5-ReadAdapter] info: Default reads: 12000, Remaining reads: 12000, Window start time: 2022-03-31T20:03:15.486679300, Window width: PT0S, Throttler time: PT0.3S

Answer 1 · 2022-05-17T07:14:20.000Z

Yes it does! I'm running the same version (0.9.9) - BTW 0.9.10 was released last week. In the Add Image dialog, just select Use existing virtual machine for Deployment.

As for authentication, your service principal needs to be in the Contributor role for the subscription.

However, I found that it's really difficult to trigger updating the list of available VMs once you've done that, essentially I had to restart the TC server to get the list updated reliably. I guess it's really aggressively caching it or something, although it looks like that it's communicating with Azure.

The agent VMs are manually provisioned using Ansible. In their buildAgent.properties file, I've set the azure.instance.name to the full name of the VM in Azure.

That's correct, but just to be sure, I'll mention that the serverUrl property also needs to be set.

Answer 2 · 2022-05-17T13:14:27.000Z

In the Add Image dialog, just select Use existing virtual machine for Deployment.

I don't recall seeing that option, but I'll double-check.

I'll mention that the serverUrl property also needs to be set.

Yes, that is set. These agents are already connected to Teamcity. I just want Teamcity to be able to start/stop them automatically.

Answer 3 · 2022-05-17T13:16:47.000Z

In the Add Image dialog, just select Use existing virtual machine for Deployment.

I don't recall seeing that option, but I'll double-check.

Oh wow, there it is. For some reason, I never checked that dropdown before. The default option of "Use specific resource group" seemed right to me, so I never tried to change it.

Thanks!