Unable to connect to Database
Closed this issue · 15 comments
Hello,
I am trying to deploy the TCA.
When I am trying to run the setup.sh file I am getting the error:
File "kg_utils.py", line 1062
logging.error(f'{e}: Issue connecting to db. Please check whether the .db file exists.')
I checked all the files that may be causing the issue.
Unable to find the db_file variable.
Please let me know how to resolve it.
Also need to have some clarity on TCA. Backend is deployed successfully and able to see UI.
Please provide deatil documentation of how to use it.
@Sayali1638 Can you provide steps to reproduce this?
If I clone the repo and create a Python3 environment and run setup.sh it works. When I run run.sh it builds the docker image and deploys it with docker-compose.
Here are the steps I followed on macOS:
git clone https://github.com/konveyor/tackle-container-advisor.git
cd tackle-container-advisor
python3 -m venv venv
source venv/bin/activate
sh setup.shThat completed successfully for me.
What steps did you take? Are you on Mac, Windows, or Linux?
@rofrano I am using Linux Ubuntu VM.
The above steps worked for me. But when ran run.sh file getting following error.
ERROR: Service 'aca_backend' failed to build: ADD failed: file not found in build context or excluded by .dockerignore: stat model_objects: file does not exist
Tried trouble shooting it but did not get the file.
Please let me know the steps to follow.
If model_objects does not exist then it sounds like setup.sh failed because that's how those objects are created.
I tried in an Linux Ubuntu 20.04 VM and sh setup.sh failed on line 10 with invalid syntax. When I used bash it worked perfectly. Try running:
bash setup.shThen check that the models have been created with: ls -l aca_backend_api/model_objects/
$ ls -l aca_backend_api/model_objects/
total 712
-rw-r--r-- 1 vagrant vagrant 264921 Feb 17 16:20 standardization_dict.pickle
-rw-r--r-- 1 vagrant vagrant 76557 Feb 17 16:20 standardization_model.pickle
-rw-r--r-- 1 vagrant vagrant 382921 Feb 17 16:20 standardization_vectorizer.pickleIf you see the .pickle files then try bash run.sh and see if it works.
Ran ./setup.sh file and got following output.
(venv) root@proofdock-testing:~/tackle-container-advisor# ./setup.sh
**Got the above output. Not sure what does the last line indicate. In video the output seems to be different.
And when I run ./run.sh I get following output.**
(venv) root@proofdock-testing:~/tackle-container-advisor# ./run.sh
-----------Running the Backend API---------
Error: No such container: aca_backend_api
Building aca_backend
Step 1/18 : FROM registry.access.redhat.com/ubi8/python-38
---> bebf43eee127
Step 2/18 : USER 0
---> Using cache
---> 39b2c8533b87
Step 3/18 : WORKDIR /app
---> Using cache
---> c09ab577982c
Step 4/18 : ADD ./service /app/service
---> Using cache
---> 65e25f6f4510
Step 5/18 : ADD ./config.py /app/config.py
---> Using cache
---> 0cfa1bd1fef1
Step 6/18 : ADD ./planner.py /app/planner.py
---> Using cache
---> c7a14059a5ef
Step 7/18 : ADD ./multiprocessing_mapreduce.py /app/multiprocessing_mapreduce.py
---> Using cache
---> bd802a4ab704
Step 8/18 : ADD ./model_objects /app/model_objects
ERROR: Service 'aca_backend' failed to build: ADD failed: file not found in build context or excluded by .dockerignore: stat model_objects: file does not exist
Please let me know if anything is missing.
tca_logs_1802.log
PFA Log attached
Thank you for the log file. It was extremely useful in understanding what is happening. I still don't understand why it's happening for you and not me. This is what I've determined.
According to the end of your log:
-----------Copying KG Utility Files to Backend API------------------------
-----------Folder does not exist. Please check with Admins.---
This block of code from lines 85 - 92 in setup.sh is failing:
echo "-----------Copying KG Utility Files to Backend API------------------------"
if [ -d "aca_backend_api/ontologies/" ]; then
cp aca_kg_utils/ontologies/*.json aca_backend_api/ontologies/.
else
echo "-----------Folder does not exist. Please check with Admins.---"
exit 0
fi
echo "-----------Copied KG Utility Files to Backend API------------------------"It fails because apparently on your machine, the aca_backend_api/ontologies/ folder does not exist.
This folder is created in the step before this in setup.sh by the call to ./aca_kg_utils/kg_utils.py which defines this folder on line 150 and creates it on lines 155-156 of kg_utils.py:
150 dst_pth = fr'aca_backend_api/{config_obj["kg"]["ontologies"]}'
...
155 if not os.path.isdir(dst_pth) :
156 os.mkdir(dst_pth)I cannot explain why this code works on my machine and not on yours. I see from your logs that you are running as root and I am not. That shouldn't make a difference.
The folder name is coming from the ./aca_kg_utils/config.ini file. Did that get changed by any chance?
Could there be some permissions on your machine that is preventing kg_utils.py from programmatically creating a folder? I'm really at a loss to understand why this doesn't work for you. Maybe try manually creating the aca_backend_api/ontologies folder and run bash setup.sh again and see if that helps. I can also provide you with a Vagrantfile that you can use with Vagrant and VirtualBox to build the exact same VM that I am using if that would help.
The log at the end of setup.sh should end with:
-----------Set up for Tackle Containerzation Adviser Completed !!!---------
If you don't see that... nothing else will work.
Hello,
It worked for cpu>1.
When I am trying to run the container assessment i am getting the internal server error.
Is there any specific reason for it?
HI @Sayali1638 , can you specify the Payload json input format you used?
@Sayali1638 , I'm getting the same error , let's check what is wrong.
@lamwassi :
The payload json format is as follows which I have taken the reference from youtube video:
Is it the right format?
@lamwassi It looks like perhaps some Python dependencies got updated and are now not compatible with the code. This is the log output from the 500 error that I received. The error is ModuleNotFoundError: No module named 'scipy.sparse._csr' which is caused by line 54 of __init__ in file /app/service/sim_applier.py:
self.tfs = pickle.load(open(model_name, "rb"), encoding="utf8")
BTW, I just added support for creating a VirtualBox VM using Vagrant and I was able to build and run the service fine using that if you run it from the command line so I believe that it's the docker container build that is pulling in the wrong libraries.
Here are the Logs
[2022-03-03 18:51:55 +0000] [1] [INFO] Starting gunicorn 20.0.0
[2022-03-03 18:51:55 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
[2022-03-03 18:51:55 +0000] [1] [INFO] Using worker: threads
[2022-03-03 18:51:55 +0000] [13] [INFO] Booting worker with pid: 13
[2022-03-03 18:51:56 +0000] [14] [INFO] Booting worker with pid: 14
[2022-03-03 18:51:56,106] INFO in __init__: **********************************************************************
[2022-03-03 18:51:56,107] INFO in __init__: ************************ Runtime Graph API *************************
[2022-03-03 18:51:56,107] INFO in __init__: **********************************************************************
[2022-03-03 18:51:56,159] INFO in __init__: **********************************************************************
[2022-03-03 18:51:56,159] INFO in __init__: ************************ Runtime Graph API *************************
[2022-03-03 18:51:56,159] INFO in __init__: **********************************************************************
config ['RBAC', 'filepaths', 'NA_VALUES', 'Logs', 'Thresholds', 'Reason_Codes', 'Performance']
config ['RBAC', 'filepaths', 'NA_VALUES', 'Logs', 'Thresholds', 'Reason_Codes', 'Performance']
config ['RBAC', 'filepaths', 'NA_VALUES', 'Logs', 'Thresholds', 'Reason_Codes', 'Performance']
WARNING:root:auth_url: https://rbac-dev.nextgen-ose-85ee131ed8e71cabc202e5781fab5c58-0000.eu-de.containers.appdomain.cloud/api/v2/access?client=api&action_name=rpt%3Aview-analytics
config ['RBAC', 'filepaths', 'NA_VALUES', 'Logs', 'Thresholds', 'Reason_Codes', 'Performance']
config ['RBAC', 'filepaths', 'NA_VALUES', 'Logs', 'Thresholds', 'Reason_Codes', 'Performance']
config ['RBAC', 'filepaths', 'NA_VALUES', 'Logs', 'Thresholds', 'Reason_Codes', 'Performance']
WARNING:root:auth_url: https://rbac-dev.nextgen-ose-85ee131ed8e71cabc202e5781fab5c58-0000.eu-de.containers.appdomain.cloud/api/v2/access?client=api&action_name=rpt%3Aview-analytics
[2022-03-03 18:55:52,126] ERROR in app: Exception on /containerization-assessment [POST]
Traceback (most recent call last):
File "/opt/app-root/lib64/python3.8/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/opt/app-root/lib64/python3.8/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/opt/app-root/lib64/python3.8/site-packages/flask_restplus/api.py", line 325, in wrapper
resp = resource(*args, **kwargs)
File "/opt/app-root/lib64/python3.8/site-packages/flask/views.py", line 88, in view
return self.dispatch_request(*args, **kwargs)
File "/opt/app-root/lib64/python3.8/site-packages/flask_restplus/resource.py", line 44, in dispatch_request
resp = meth(*args, **kwargs)
File "/opt/app-root/lib64/python3.8/site-packages/flask_restplus/marshalling.py", line 243, in wrapper
resp = f(*args, **kwargs)
File "/app/service/routes.py", line 129, in post
return planner.do_assessment(auth_url,dict(request.headers),auth_headers,api.payload)
File "/app/planner.py", line 235, in do_assessment
controller = Planner()
File "/app/planner.py", line 43, in __init__
self.entity_detection = EntityDetection()
File "/app/service/entity_detection.py", line 69, in __init__
self.__sim = sim_applier()
File "/app/service/sim_applier.py", line 54, in __init__
self.tfs = pickle.load(open(model_name, "rb"), encoding="utf8")
ModuleNotFoundError: No module named 'scipy.sparse._csr'
ERROR:flask.app:Exception on /containerization-assessment [POST]
Traceback (most recent call last):
File "/opt/app-root/lib64/python3.8/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/opt/app-root/lib64/python3.8/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/opt/app-root/lib64/python3.8/site-packages/flask_restplus/api.py", line 325, in wrapper
resp = resource(*args, **kwargs)
File "/opt/app-root/lib64/python3.8/site-packages/flask/views.py", line 88, in view
return self.dispatch_request(*args, **kwargs)
File "/opt/app-root/lib64/python3.8/site-packages/flask_restplus/resource.py", line 44, in dispatch_request
resp = meth(*args, **kwargs)
File "/opt/app-root/lib64/python3.8/site-packages/flask_restplus/marshalling.py", line 243, in wrapper
resp = f(*args, **kwargs)
File "/app/service/routes.py", line 129, in post
return planner.do_assessment(auth_url,dict(request.headers),auth_headers,api.payload)
File "/app/planner.py", line 235, in do_assessment
controller = Planner()
File "/app/planner.py", line 43, in __init__
self.entity_detection = EntityDetection()
File "/app/service/entity_detection.py", line 69, in __init__
self.__sim = sim_applier()
File "/app/service/sim_applier.py", line 54, in __init__
self.tfs = pickle.load(open(model_name, "rb"), encoding="utf8")
ModuleNotFoundError: No module named 'scipy.sparse._csr'
How do I resolve the error at my end? I am not able to understand what has gone wrong. Where can I find the logs?
Request you please evaluate.
@Sayali1638 There is nothing you can do at your end. The problem is with the Python dependencies in the code. I've submitted this Pull Request to hopefully fix it.
@lamwassi Can you see if this PR fixes it for you and if so, please merge and let @Sayali1638 know to try again.
@rofrano , the Backend API now runs correctly. @Sayali1638 you can now pull the latest code and retry it. Also we provided the steps to run the backend in our READ.ME file
resolved