Azure/Stormspotter

Error in upload

legra-ms opened this issue · 11 comments

Having the same issue also. Stormcollector ran successfully but unable uploading to the FE wasn't successful
image

Originally posted by @davidokeyode in #21 (comment)

I can send the output file to you if that helps. It's a test environment that will be gone in a few days so nothing sensitive.

That works. You can post the file here if you're fine with that.

That works. You can post the file here if you're fine with that.
Added. Cheers.
results_20200914-175350.zip
stormresult.zip

@davidokeyode I was able to successfully upload your results file on the first try.
image

image

Can you provide your docker logs?

@davidokeyode I was able to successfully upload your results file on the first try.
image

image

Can you provide your docker logs?

How do I get the logs? Thanks.

When you run docker-compose up, you should see all your logs in the same terminal you ran the command. If you're using something like Docker for Windows to start and stop your containers, then you can access the logs through the GUI.

image

When you run docker-compose up, you should see all your logs in the same terminal you ran the command. If you're using something like Docker for Windows to start and stop your containers, then you can access the logs through the GUI.

image

Yeah I don't see any output when uploading the file (the logs below is from the output after running docker-compose but nothing afterwards)

stormspotter-frontend_1  | GET /dashboard [86.181.142.97] Mon Sep 14 2020 17:51                               :05 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /css/vendor.a6db43f1.css [86.181.142.97] Mon Sep                                14 2020 17:51:05 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /css/app.2eda0ddd.css [86.181.142.97] Mon Sep 14                                2020 17:51:05 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /js/vendor.f2ec0c5f.js [86.181.142.97] Mon Sep 1                               4 2020 17:51:05 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /js/app.871708e6.js [86.181.142.97] Mon Sep 14 2                               020 17:51:05 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /css/3.af5418f4.css [86.181.142.97] Mon Sep 14 2                               020 17:51:06 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /js/3.7174c868.js [86.181.142.97] Mon Sep 14 202                               0 17:51:06 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /logo.png [86.181.142.97] Mon Sep 14 2020 17:51:                               06 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /clouds.mp4 [86.181.142.97] Mon Sep 14 2020 17:5                               1:06 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /fonts/flUhRq6tzZclQEJ-Vdg-IuiaDsNcIhQ8tQ.fa3334                               fe.woff2 [86.181.142.97] Mon Sep 14 2020 17:51:06 GMT+0000 (Coordinated Univers                               al Time)
stormspotter-frontend_1  | GET /fonts/CascadiaMono.e6d823d0.ttf [86.181.142.97]                                Mon Sep 14 2020 17:51:06 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /clouds.mp4 [86.181.142.97] Mon Sep 14 2020 17:5                               1:06 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /favicon.ico [86.181.142.97] Mon Sep 14 2020 17:                               51:06 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /clouds.mp4 [86.181.142.97] Mon Sep 14 2020 17:5                               1:07 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /js/2.1ec371a8.js [86.181.142.97] Mon Sep 14 202                               0 17:51:24 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /css/2.62377c4f.css [86.181.142.97] Mon Sep 14 2                               020 17:51:24 GMT+0000 (Coordinated Universal Time)

















stormspotter-frontend_1  | GET /dashboard [86.181.142.97] Mon Sep 14 2020 18:39                               :33 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /clouds.mp4 [86.181.142.97] Mon Sep 14 2020 18:3                               9:35 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /clouds.mp4 [86.181.142.97] Mon Sep 14 2020 18:3                               9:35 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /clouds.mp4 [86.181.142.97] Mon Sep 14 2020 18:4                               0:03 GMT+0000 (Coordinated Universal Time)
^CGracefully stopping... (press Ctrl+C again to force)
Stopping stormspotter_stormspotter-frontend_1 ...
Stopping stormspotter_stormspotter-backend_1  ...
Stopping stormspotter_stormspotter-neo4j_1    ...
Killing stormspotter_stormspotter-frontend_1  ... done
Killing stormspotter_stormspotter-backend_1   ... error
Killing stormspotter_stormspotter-neo4j_1     ... done

ERROR: for stormspotter_stormspotter-backend_1  Cannot kill container: 486199ff                               22c778ced517c11f1df74722bf49aff2813af3f08b3be59ebeb31b48: Container 486199ff22c                               778ced517c11f1df74722bf49aff2813af3f08b3be59ebeb31b48 is not running
root@ubuntuvm01:~/Stormspotter# docker-compose up
Starting stormspotter_stormspotter-neo4j_1 ... done
Starting stormspotter_stormspotter-frontend_1 ... done
Starting stormspotter_stormspotter-backend_1  ... done
Attaching to stormspotter_stormspotter-neo4j_1, stormspotter_stormspotter-front                               end_1, stormspotter_stormspotter-backend_1
stormspotter-frontend_1  |
stormspotter-frontend_1  |  Quasar CLI.......... v1.1.0
stormspotter-frontend_1  |  Listening at........ http://9c04246d041e:9091
stormspotter-frontend_1  |  Web server root..... /usr/src/app
stormspotter-frontend_1  |  Gzip................ enabled
stormspotter-frontend_1  |  Cache (max-age)..... 86400
stormspotter-frontend_1  |  Micro-cache......... 1s
stormspotter-frontend_1  |  History mode........ enabled
stormspotter-frontend_1  |  Index file.......... index.html
stormspotter-frontend_1  |
stormspotter-neo4j_1     | Warning: Some files inside "/data" are not writable                                from inside container. Changing folder owner to neo4j.
stormspotter-neo4j_1     | WARNING: Max 1024 open files allowed, minimum of 400                               00 recommended. See the Neo4j manual.
stormspotter-neo4j_1     | Changed password for user 'neo4j'.
stormspotter-neo4j_1     | Directories in use:
stormspotter-neo4j_1     |   home:         /var/lib/neo4j
stormspotter-neo4j_1     |   config:       /var/lib/neo4j/conf
stormspotter-neo4j_1     |   logs:         /logs
stormspotter-neo4j_1     |   plugins:      /var/lib/neo4j/plugins
stormspotter-neo4j_1     |   import:       /var/lib/neo4j/import
stormspotter-neo4j_1     |   data:         /var/lib/neo4j/data
stormspotter-neo4j_1     |   certificates: /var/lib/neo4j/certificates
stormspotter-neo4j_1     |   run:          /var/lib/neo4j/run
stormspotter-neo4j_1     | Starting Neo4j.
stormspotter-neo4j_1     | WARNING: Max 1024 open files allowed, minimum of 400                               00 recommended. See the Neo4j manual.
stormspotter-backend_1   | INFO:     Started server process [1]
stormspotter-backend_1   | INFO:     Waiting for application startup.
stormspotter-backend_1   | INFO:     Application startup complete.
stormspotter-backend_1   | INFO:     Uvicorn running on http://0.0.0.0:9090 (Pr                               ess CTRL+C to quit)
stormspotter-neo4j_1     | 2020-09-14 18:42:52.198+0000 INFO  Starting...
stormspotter-neo4j_1     | 2020-09-14 18:42:55.530+0000 INFO  ======== Neo4j 4.                               1.1 ========
stormspotter-neo4j_1     | 2020-09-14 18:42:57.349+0000 INFO  Performing postIn                               itialization step for component 'security-users' with version 2 and status CURR                               ENT
stormspotter-neo4j_1     | 2020-09-14 18:42:57.350+0000 INFO  Updating the init                               ial password in component 'security-users'
stormspotter-neo4j_1     | 2020-09-14 18:42:57.359+0000 INFO  Updating initial                                user password from `auth.ini` file: neo4j
stormspotter-neo4j_1     | 2020-09-14 18:42:57.713+0000 INFO  Bolt enabled on 0                               .0.0.0:7687.
stormspotter-neo4j_1     | 2020-09-14 18:43:00.124+0000 INFO  Remote interface                                available at http://localhost:7474/
stormspotter-neo4j_1     | 2020-09-14 18:43:00.126+0000 INFO  Started.






stormspotter-frontend_1  | GET / [86.181.142.97] Mon Sep 14 2020 18:43:28 GMT+0                               000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /favicon.ico [86.181.142.97] Mon Sep 14 2020 18:                               43:29 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /dashboard [86.181.142.97] Mon Sep 14 2020 19:00                               :01 GMT+0000 (Coordinated Universal Time)
stormspotter-frontend_1  | GET /clouds.mp4 [86.181.142.97] Mon Sep 14 2020 19:0                               0:02 GMT+0000 (Coordinated Universal Time)
stormspotter-backend_1   | 2020-09-14 19:37:02.980 | INFO     | uvicorn.protoco                               ls.http.httptools_impl:send:452 - 86.181.142.97:50187 - "GET / HTTP/1.1" 404
stormspotter-backend_1   | 2020-09-14 19:37:03.348 | INFO     | uvicorn.protoco                               ls.http.httptools_impl:send:452 - 86.181.142.97:50187 - "GET /favicon.ico HTTP/                               1.1" 404

Are you hosting Stormspotter in a cloud environment or something? That IP address in the HTTP logs is a public IP so just wondering, since I haven't tested running it in Azure or anything.

Anyway, the next thing to check would be the console logs of your browser. It should tell you about any failed connections, either to neo4j or the backend containers. In this pic, I turned off the backend and tried uploading the file:

image

Also, it appears that you might have a bad container dangling that might be affecting the upload? See if you can kill the bad container before trying to docker-compose up.

stormspotter_stormspotter-backend_1 Cannot kill container: 486199ff

Are you hosting Stormspotter in a cloud environment or something? That IP address in the HTTP logs is a public IP so just wondering, since I haven't tested running it in Azure or anything.

Anyway, the next thing to check would be the console logs of your browser. It should tell you about any failed connections, either to neo4j or the backend containers. In this pic, I turned off the backend and tried uploading the file:

image

I see the issue now. Yes I am hosting it on an Azure VM but the upload script is targetting localhost:9090. I'll run locally. Thanks for your help. On a side note, it might be good to update the README to highlight that it should only be ran locally to avoid the same issue. Cheers again.
image

Thanks for the patience and glad we figured it out.

For sure, I'll add that to the README. When I first starting dockerizing this, I realized there would be a bunch of issues (this was the first issue) if run in a cloud. Some of the others would be handling security (i.e. authentication) to the backend and neo4j endpoints.

I originally tried to include the frontend and the backend in the same container, served by nginx where the backend was located at an /api path but I was unsuccessful.

If I can eventually get that to work, then this shouldn't be an issue in the future. For now, I'll close this, create a feature issue to work on this in the future, and update the README.