Environment variables not found

Question

Environment variables not found

raamanihilent opened this issue 6 years ago · 27 comments

raamanihilent commented 6 years ago

Type of issue

Bug
Tried following the document at
https://docs.microsoft.com/en-us/azure/iot-accelerators/iot-accelerators-remote-monitoring-deploy-local-docker
New feature
Improvement

Description

docker-compose up command throws the below error

Screenshot

Steps to reproduce

[First step]
[Second step]
[and so on...]

Expected behavior

...

Current behavior

...

Known workarounds

...

Possible solution

...

Context and Environment

Operating System: ...
GitHub branch: ...
.NET Runtime: ...

Answer 1 · 2019-05-09T21:43:24.000Z

Thanks for the update Parvez.
@sushilraje

I am listing a few more items, which i encountered during my debugging by starting the services separately in a command prompt by using the run.cmd inside docker directory.

The container APIs are accessible from the browser. However they are not reached by the containers.
Some of the expected collections and values are not present in the Cosmos DB.
http://localhost:9022/v1/collections/device-twin-properties/values/cache
http://localhost:9022/v1/collections/solution-settings/values/seedMutex
http://127.0.0.1:9004/v1/rules
If there is no seed data injection, please provide steps to call the REST APIs(headers/body)

Hope this helps.
Thanks

Answer 2 · 2019-05-10T22:32:49.000Z

@raamanihilent: I believe you need to run the pcs -s local command after pcs login but before docker-compose up. Please run it and let me know if it works.

Answer 3 · 2019-05-10T22:53:44.000Z

@sushilraje
Thanks.
It did not help.
when i run the pcs -s local command, the command prompts to create Azure resources, which was already done by executing the start.cmd command from the documentation link provided earlier.

The following steps were followed and i could verify that the azure resources were created.
Run the start.cmd script. The script prompts you for the following information:
A solution name.
The Azure subscription to use.
The location of the Azure datacenter to use.

Will wait for your updates. Ty.

Answer 4 · 2019-05-11T01:17:57.000Z

@raamanihilent:
It seems that start.cmd didn't stamp the environment variables on your machine. I will try to repro this issue. But a quick fix would be to create following environment variables on your machine.
PCS_KEYVAULT_NAME
PCS_AAD_APPID
PCS_AAD_APPSECRET
Values for these variables would be
Go to your azure portal go to resource group (same as solution name)
keyvault-****
go to keyvault > secrets
aadAppId
aadAppSecret

Answer 5 · 2019-05-11T01:36:02.000Z

@sushilraje
Thanks for the information.
It was set correctly by start.cmd command. I have verified this before creating the issue.
Here is my env snapshot.

Will wait for your updates.

Answer 6 · 2019-05-13T18:08:25.000Z

@raamanihilent Which docker compose yaml file are you using?
azure-iot-pcs-remote-monitoring-dotnet\services\scripts\local OR
azure-iot-pcs-remote-monitoring-dotnet\scripts\local

You should be using the one underneath the services folder.

Answer 7 · 2019-05-13T20:16:33.000Z

yes I ran the script from
azure-iot-pcs-remote-monitoring-dotnet\services\scripts\local

Answer 8 · 2019-05-14T17:30:50.000Z

Can you check foll,?
Does your docker-compose.yml file have foll structure,

especially the highlighted environment variables.

:
image: azureiotpcs/-dotnet:testing
.........
environment:
- PCS_KEYVAULT_NAME
- PCS_AAD_APPID
- PCS_AAD_APPSECRET

Also, could you try deleting all the images from your local machine and try docker-compose up?

Answer 9 · 2019-05-14T21:52:42.000Z

Yes and i deleted all the images, ran docker-compose up.
Did not work .
Will wait for your updates.

Answer 10 · 2019-05-16T18:04:22.000Z

Can you docker inspect on any of the containers and ensure that the variables are correct values are passed through?

Answer 11 · 2019-05-17T00:59:09.000Z

I tried it out locally, it did work for me.
Please do docker inspect and check if the required three variables
PCS_AAD_APPID,
PCS_AAD_APPSECRET,
KEYVAULT_NAME
are getting passed along correctly as environment variables to the container.
Also, what is the tag of your containers?

Answer 12 · 2019-05-17T14:44:55.000Z

Thanks Let me inspect and get back.

Answer 13 · 2019-05-21T18:08:02.000Z

@raamanihilent: Were you able to get any data around the issue?

Answer 14 · 2019-05-21T20:28:41.000Z

@sushilraje Since you mentioned it worked for you locally, I am planning to redo the steps. Let me get back to you with my updates.

Answer 15 · 2019-05-22T21:03:49.000Z

@sushilraje
As mentioned i cleaned up my environment.
Followed the steps again and tried docker-compose up. The local proxy was failing with error -

ERROR: for reverseproxy Cannot start service reverseproxy: OCI runtime create failed: container_linux.go:344: starting container process caused "process_linux.go:424: container init caused "rootfs_linux.go:58: mounting \"/host_mnt/c/MyStuff/remotemonitoring/azure-iot-pcs-remote-monitoring-dotnet/services/scripts/local/config/nginx.conf\" to rootfs \"/var/lib/docker/overlay2/f9531ebe0054b7d0436008af3945903e9f311f3052d029ab202d0ef3dba0cbd7/merged\" at \"/var/lib/docker/overlay2/f9531ebe0054b7d0436008af3945903e9f311f3052d029ab202d0ef3dba0cbd7/merged/app/config/nginx.conf\" caused \"not a directory\""": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
ERROR: Encountered errors while bringing up the project.

I then stopped all my containers, started each container in a separate window and inspected them.
I could verify the following environment variables set via docker inspect
"PCS_KEYVAULT_NAME=",
"PCS_AAD_APPID=",
"PCS_AAD_APPSECRET=",

However these are my errors in console -
Config container

StorageAdapter returns 0 for request http://localhost:9022/v1/collections/device-twin-properties/values/cache, {"Uri":"http://localhost:9022/v1/collections/device-twin-properties/values/cache","StatusCode":0,"Content":"An error occurred while sending the request. - Couldn't connect to server"}

ASA

Received error updating device to device group mapping, {"e":{"ExceptionFullName":"Microsoft.Azure.IoTSolutions.AsaManager.Services.Exceptions.ExternalDependencyException","ExceptionMessage":"Unable to get list of device groups","StackTrace":" at Microsoft.Azure.IoTSolutions.AsaManager.DeviceGroupsAgent.DeviceGroupsClient.d__8.MoveNext() in /home/azureiotpcs/vsts-agent/_work/20/s/asa-manager/DeviceGroupsAgent/DeviceGroupsClient.cs:line 83\n--- End of stack trace from previous location where exception was thrown ---\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\n at Microsoft.Azure.IoTSolutions.AsaManager.DeviceGroupsAgent.Agent.d__13.MoveNext() in /home/azureiotpcs/vsts-agent/_work/20/s/asa-manager/DeviceGroupsAgent/Agent.cs:line 85","Source":"Microsoft.Azure.IoTSolutions.AsaManager.DeviceGroupsAgent","Data":{},"InnerException":{"ExceptionFullName":"Microsoft.Azure.IoTSolutions.AsaManager.Services.Exceptions.ExternalDependencyException","ExceptionMessage":"Unable to load get device groups","StackTrace":" at Microsoft.Azure.IoTSolutions.AsaManager.Services.Http.HttpClient.d__9`1.MoveNext() in /home/azureiotpcs/vsts-agent/_work/20/s/asa-manager/Services/Http/HttpClient.cs:line 104\n--- End of stack trace from previous location where exception was thrown ---\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\n at Microsoft.Azure.IoTSolutions.AsaManager.DeviceGroupsAgent.DeviceGroupsClient.d__8.MoveNext() in /home/azureiotpcs/vsts-agent/_work/20/s/asa-manager/DeviceGroupsAgent/DeviceGroupsClient.cs:line 78","Source":"Microsoft.Azure.IoTSolutions.AsaManager.Services","Data":{},"InnerException":null}}}

iot_hub_manager

[StorageAdapterClient:CheckStatusCode] StorageAdapter returns 0 for request http://localhost:9022/v1/collections/device-twin-properties/values/cache, {"Uri":"http://localhost:9022/v1/collections/device-twin-properties/values/cache","StatusCode":0,"Content":"An error occurred while sending the request. - Couldn't connect to server"}

Will wait for your updates.
Thanks

Answer 16 · 2019-05-29T04:31:00.000Z

deploy locally with Visual studio - cannot start the microservices as well - any leads on this ?

environment variables set correctly -

any more changes here ? please advise, thanks

Answer 17 · 2019-05-29T04:58:44.000Z

Also can you provide more details for the deploy with VS Code -
Deploy all other microservices on local machine
Open the azure-iot-pcs-remote-monitoring-dotnet module from your local copy in VS Code.
Copy the files launch.json and tasks.json from scripts\local\launch\idesettings\vscode. Create a new folder azure-iot-pcs-remote-monitoring-dotnet.vscode and paste the files there.
Open the Debug panel in VS Code and run the Run all microservices configuration. This configuration runs the device simulation microservice in Docker and runs the other microservices in the debugger.
don't see an option in debug panel for running all microservices... can you please provide detailed steps for this. Thanks

Answer 18 · 2019-05-29T06:06:40.000Z

The microservices don't start when doing a docker compose up either - screenshot below
Can you please add on more steps here to have it running. Thanks.

Answer 19 · 2019-06-03T23:04:13.000Z

@raamanihilent @ac2707 I am extremely sorry for the delayed response.

Answer 20 · 2019-06-03T23:04:31.000Z

@raamanihilent: We do need to mount a directory containing nginx.conf file on to the docker container.
If you are using Windows, you will need to give permissions to docker daemon.
Open your docker console > settings > Shared Drives > Select your drives > Apply

Answer 21 · 2019-06-03T23:14:05.000Z

@ac2707 sn

deploy locally with Visual studio - cannot start the microservices as well - any leads on this ?

environment variables set correctly -

any more changes here ? please advise, thanks

This is actually working. The logs are not Error level but warnings. Here microservices are trying to inherit env variables. As a first step, the services try to inherit files from local env, if not found then the services go to key-vault. Since you have given key-vault credentials, the services will get the variables/secrets from key-vault.

Answer 22 · 2019-06-03T23:31:17.000Z

Also can you provide more details for the deploy with VS Code -
Deploy all other microservices on local machine
Open the azure-iot-pcs-remote-monitoring-dotnet module from your local copy in VS Code.
Copy the files launch.json and tasks.json from scripts\local\launch\idesettings\vscode. Create a new folder azure-iot-pcs-remote-monitoring-dotnet.vscode and paste the files there.
Open the Debug panel in VS Code and run the Run all microservices configuration. This configuration runs the device simulation microservice in Docker and runs the other microservices in the debugger.
don't see an option in debug panel for running all microservices... can you please provide detailed steps for this. Thanks

In your file launch.json from scripts\local\launch\idesettings\vscode, do you see "compound" section with name "Run all Microservices" as here?

Answer 23 · 2019-06-03T23:37:36.000Z

The microservices don't start when doing a docker compose up either - screenshot below
Can you please add on more steps here to have it running. Thanks.

This is a different issue than just provisioning. Can you open up a new issue? In that please add foll. details

o/p of docker ps
o/p of docker ps -a
Complete logs of
1. UiConfig/Config
2. Device Simulation
3. Storage Adapter microservices,
  preferably as files

Answer 24 · 2019-06-06T22:33:14.000Z

@raamanihilent & @ac2707 Do let me know if you face any further issues.

Answer 25 · 2019-06-07T05:17:55.000Z

The microservices don't start when doing a docker compose up either - screenshot below
Can you please add on more steps here to have it running. Thanks.

This is a different issue than just provisioning. Can you open up a new issue? In that please add foll. details

o/p of docker ps

o/p of docker ps -a

Complete logs of

UiConfig/Config

Device Simulation

Storage Adapter microservices,
preferably as files

Device simulation --

Answer 26 · 2019-06-07T05:33:39.000Z

@ac2707 sn

deploy locally with Visual studio - cannot start the microservices as well - any leads on this ?

environment variables set correctly -

any more changes here ? please advise, thanks

This is actually working. The logs are not Error level but warnings. Here microservices are trying to inherit env variables. As a first step, the services try to inherit files from local env, if not found then the services go to key-vault. Since you have given key-vault credentials, the services will get the variables/secrets from key-vault.

The command prompts and the web browser windows close, unable to run the microservices from Visual studio..
debug output log in Visual studio below -

Answer 27 · 2019-10-06T08:07:21.000Z

Any Solution Found for starting microservices ???