firefart/network_info

I tried to run your script

ozkanpakdil opened this issue · 5 comments

ozkanpakdil commented

I installed everything accordingly
apt-get install python3-sqlalchemy python3-netaddr python3-psycopg2

and configured postgresql like this otherwise I was having no password errors.
local all all trust
host all all 127.0.0.1/32 trust
host all all ::1/128 trust
host all all localhost trust

and run the script with this command "python3 create_ripe_db.py" after long parsing info logs it finished with these lines
2016-11-27 11:52:41,059 - create_ripe_db - INFO - MainProcess - block parsing finished: 998.06 seconds
2016-11-27 11:52:41,075 - create_ripe_db - INFO - MainProcess - parsing IPs
2016-11-27 11:52:41,110 - create_ripe_db - DEBUG - MainProcess - starting 8 processes
2016-11-27 11:52:41,398 - create_ripe_db - DEBUG - MainProcess - populating job queue
Killed

my system is linux mint which is like ubuntu clone. and hardware is 8GB ram and i7 2GHZ. block table has 12175 rows and cidr is empty. I wonder what is killed and if this is a problem how can I pass this ?

thanks

firefart commented

@ozkanpakdil maybe you ran out of memory or smth. in mint killed to process to keep the system stable. You can try to manually decrease the NUM_WORKERS and/or COMMIT_COUNT. This might help to reduce system load

ozkanpakdil commented

@firefart I tried to decreasing those parameters but result did not change. somehow code fills "block" table but could not fill cidr table. I tried to change the code just to fill cidr but could not manage it. I dont know python much. I am sure there is something wrong about thread management. because after filling block table. it waits for a while until it gives "Killed". and code does not eat any cpu in that waiting time.

thanks

firefart commented

it works without problems here maybe you changed too much code? Also be sure you have all requirements installed by pip install -r requirements.txt

formtapez commented

I ran into the same problem with that missing password like ozkanpakdil does. (On Ubuntu 16.04)
Therefor i changed the URL in helper.py to 'postgresql://ripe:ripe@localhost/ripe' and created the database with a password. Everything went fine then.

firefart commented

@ozkanpakdil the master branch has been updated with a new version and a docker image. Be sure to check it out if you are still experiencing problems.