iotedge deployment issues with ML module

Question

iotedge deployment issues with ML module

payalgaikwad42 opened this issue 6 years ago · 4 comments

i have deployed machine learning module on iotedge but unable to see module running when entered the command iotedge list
when i do docker ps it shows me the container is up and logs of the container are as follows:

edgeAgent
PS C:\WINDOWS\system32> docker logs -f hungry_lalande
2018-08-13 10:02:02,233 CRIT Supervisor running as root (no user in config file)
2018-08-13 10:02:02,236 INFO supervisord started with pid 1
2018-08-13 10:02:03,268 INFO spawned: 'rsyslog' with pid 7
2018-08-13 10:02:03,271 INFO spawned: 'program_exit' with pid 8
2018-08-13 10:02:03,274 INFO spawned: 'nginx' with pid 9
2018-08-13 10:02:03,284 INFO spawned: 'iot' with pid 10
2018-08-13 10:02:03,289 INFO spawned: 'gunicorn' with pid 11
2018-08-13 10:02:03,972 INFO success: iot entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
EdgeHubConnectionString and IOTEDGE_IOTHUBHOSTNAME are not set. Exiting...
2018-08-13 10:02:04,486 INFO success: rsyslog entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-08-13 10:02:04,486 INFO success: program_exit entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-08-13 10:02:04,488 INFO exited: iot (exit status 1; expected)
2018-08-13 10:02:08,495 INFO success: nginx entered RUNNING state, process has stayed up for > than 5 seconds (startsecs)
{"logger": "gunicorn.error", "host": "becfcda46091", "timestamp": "2018-08-13T10:02:09.857704Z", "level": "INFO", "msg": "Starting gunicorn %s", "tags": "%(module)s, %(asctime)s, %(levelname)s, %(message)s", "path": "/home/mmlspark/lib/conda/lib/python3.5/site-packages/gunicorn/glogging.py", "stack_info": null, "message": "Starting gunicorn 19.6.0"}
{"logger": "gunicorn.error", "host": "becfcda46091", "timestamp": "2018-08-13T10:02:09.858522Z", "level": "INFO", "msg": "Listening at: %s (%s)", "tags": "%(module)s, %(asctime)s, %(levelname)s, %(message)s", "path": "/home/mmlspark/lib/conda/lib/python3.5/site-packages/gunicorn/glogging.py", "stack_info": null, "message": "Listening at: http://127.0.0.1:9090 (11)"}
{"logger": "gunicorn.error", "host": "becfcda46091", "timestamp": "2018-08-13T10:02:09.858623Z", "level": "INFO", "msg": "Using worker: %s", "tags": "%(module)s, %(asctime)s, %(levelname)s, %(message)s", "path": "/home/mmlspark/lib/conda/lib/python3.5/site-packages/gunicorn/glogging.py", "stack_info": null, "message": "Using worker: sync"}
{"logger": "gunicorn.error", "host": "becfcda46091", "timestamp": "2018-08-13T10:02:09.859228Z", "level": "INFO", "stack_info": null, "tags": "%(module)s, %(asctime)s, %(levelname)s, %(message)s", "path": "/home/mmlspark/lib/conda/lib/python3.5/site-packages/gunicorn/glogging.py", "message": "worker timeout is set to 300"}
{"logger": "gunicorn.error", "host": "becfcda46091", "timestamp": "2018-08-13T10:02:09.860165Z", "level": "INFO", "msg": "Booting worker with pid: %s", "tags": "%(module)s, %(asctime)s, %(levelname)s, %(message)s", "path": "/home/mmlspark/lib/conda/lib/python3.5/site-packages/gunicorn/glogging.py", "stack_info": null, "message": "Booting worker with pid: 27"}
Initializing logger
{"logger": "root", "host": "becfcda46091", "timestamp": "2018-08-13T10:02:12.307585Z", "level": "INFO", "stack_info": null, "tags": "%(module)s, %(asctime)s, %(levelname)s, %(message)s", "path": "/var/azureml-app/aml_logger.py", "message": "{"requestId": "00000000-0000-0000-0000-000000000000", "message": "Starting up app insights client", "apiName": ""}"}
{"logger": "root", "host": "becfcda46091", "timestamp": "2018-08-13T10:02:12.307761Z", "level": "INFO", "stack_info": null, "tags": "%(module)s, %(asctime)s, %(levelname)s, %(message)s", "path": "/var/azureml-app/aml_logger.py", "message": "{"requestId": "00000000-0000-0000-0000-000000000000", "message": "Starting up request id generator", "apiName": ""}"}
{"logger": "root", "host": "becfcda46091", "timestamp": "2018-08-13T10:02:12.307843Z", "level": "INFO", "stack_info": null, "tags": "%(module)s, %(asctime)s, %(levelname)s, %(message)s", "path": "/var/azureml-app/aml_logger.py", "message": "{"requestId": "00000000-0000-0000-0000-000000000000", "message": "Starting up app insight hooks", "apiName": ""}"}
{"logger": "root", "host": "becfcda46091", "timestamp": "2018-08-13T10:02:12.307940Z", "level": "INFO", "stack_info": null, "tags": "%(module)s, %(asctime)s, %(levelname)s, %(message)s", "path": "/var/azureml-app/aml_logger.py", "message": "{"requestId": "00000000-0000-0000-0000-000000000000", "message": "Invoking user's init function", "apiName": ""}"}
AML_MODEL_DC_STORAGE must be set.
AML_MODEL_DC_STORAGE must be set.
{"logger": "root", "host": "becfcda46091", "timestamp": "2018-08-13T10:02:13.540737Z", "level": "INFO", "stack_info": null, "tags": "%(module)s, %(asctime)s, %(levelname)s, %(message)s", "path": "/var/azureml-app/aml_logger.py", "message": "{"requestId": "00000000-0000-0000-0000-000000000000", "message": "Users's init has completed successfully", "apiName": ""}"}
/home/mmlspark/lib/conda/lib/python3.5/site-packages/sklearn/base.py:312: UserWarning: Trying to unpickle estimator LogisticRegression from version 0.18.1 when using version 0.19.0. This might lead to breaking code or invalid results. Use at your own risk.
UserWarning)
{"logger": "logger_stderr", "request_id": "no request id", "message": "/home/mmlspark/lib/conda/lib/python3.5/site-packages/sklearn/base.py:312: UserWarning: Trying to unpickle estimator LogisticRegression from version 0.18.1 when using version 0.19.0. This might lead to breaking code or invalid results. Use at your own risk.\n UserWarning)\n", "timestamp": "2018-08-13T10:02:13.531866"}
/home/mmlspark/lib/conda/lib/python3.5/site-packages/azureml/datacollector/modeldatacollector.py:104: UserWarning: initialize failed: environment variable AML_MODEL_DC_STORAGE must be set to an Azure storage connection string (http://aka.ms/amlmodeldatacollection).
warnings.warn('initialize failed: environment variable AML_MODEL_DC_STORAGE must be set to an Azure storage connection string (http://aka.ms/amlmodeldatacollection).')
{"logger": "logger_stderr", "request_id": "no request id", "message": "/home/mmlspark/lib/conda/lib/python3.5/site-packages/azureml/datacollector/modeldatacollector.py:104: UserWarning: initialize failed: environment variable AML_MODEL_DC_STORAGE must be set to an Azure storage connection string (http://aka.ms/amlmodeldatacollection).\n warnings.warn('initialize failed: environment variable AML_MODEL_DC_STORAGE must be set to an Azure storage connection string (http://aka.ms/amlmodeldatacollection).')\n", "timestamp": "2018-08-13T10:02:13.540065"}
{"logger": "root", "host": "becfcda46091", "timestamp": "2018-08-13T10:02:13.541186Z", "level": "INFO", "stack_info": null, "tags": "%(module)s, %(asctime)s, %(levelname)s, %(message)s", "path": "/var/azureml-app/aml_logger.py", "message": "{"requestId": "00000000-0000-0000-0000-000000000000", "message": "Scoring timeout setting is not found. Use default timeout: 3600000 ms", "apiName": ""}"}
{"logger": "gunicorn.access", "host": "becfcda46091", "timestamp": "2018-08-13T10:02:13.542957Z", "level": "INFO", "stack_info": null, "tags": "%(module)s, %(asctime)s, %(levelname)s, %(message)s", "path": "/home/mmlspark/lib/conda/lib/python3.5/site-packages/gunicorn/glogging.py", "message": "127.0.0.1 - - [13/Aug/2018:10:02:13 +0000] "GET / HTTP/1.0" 200 7 "-" "python-requests/2.19.1""}
{"logger": "root", "host": "becfcda46091", "timestamp": "2018-08-13T10:02:13.557767Z", "level": "INFO", "stack_info": null, "tags": "%(module)s, %(asctime)s, %(levelname)s, %(message)s", "path": "/var/azureml-app/aml_logger.py", "message": "{"requestId": "5353f90f-e3e8-401b-9aff-0d62832556c8", "message": 200, "apiName": "/swagger.json"}"}
{"logger": "gunicorn.access", "host": "becfcda46091", "timestamp": "2018-08-13T10:02:13.558413Z", "level": "INFO", "stack_info": null, "tags": "%(module)s, %(asctime)s, %(levelname)s, %(message)s", "path": "/home/mmlspark/lib/conda/lib/python3.5/site-packages/gunicorn/glogging.py", "message": "127.0.0.1 - - [13/Aug/2018:10:02:13 +0000] "GET /swagger.json HTTP/1.0" 200 2205 "-" "python-requests/2.19.1""}

what should i do?

Answer 1 · 2018-08-15T23:42:33.000Z

Could you please copy/paste your deployment JSON (eliding secrets as appropriate)?

Answer 2 · 2018-08-16T06:20:05.000Z

i dont have deployment JSON file as such as i am using azure machine learning bench from there i am using command prompt to creat a realtime service with this command:
az ml service create realtime -f score_iris.py --model-file model.pkl -s service_schema.json -n irisapp -r python --collect-model-data true -c aml_config\conda_dependencies.yml

it creates the image and stores it in azure container registry. This stored image i am using to create ne iotedge module.

Answer 3 · 2018-08-17T05:00:23.000Z

this is my deployment.json file :
deployment.txt
i have followed this tutorial:
https://docs.microsoft.com/en-us/azure/iot-edge/tutorial-deploy-machine-learning

Answer 4 · 2018-08-30T21:09:43.000Z

Hi @payalgaikwad42 ,

Can you move your issue to https://github.com/Azure/iotedge since it's v2 related? It will get more attention there.

Closing here so you can open there.

Thanks,

Angelo Ribeiro.