Remove website-specific code

[snadi]

Is the following code in addlibraries.py needed for running the scripts?

import os
import json
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.append(BASE_DIR)
os.environ['DJANGO_SETTINGS_MODULE'] = 'metricwebsite.settings'
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "metricwebsite.settings")
import django
django.setup()

or is this specific to the website? Do we even need it for the website?

[relhajj]

Ok so I looked into it and from my understanding, the entire addlibraries.py file is not needed for running the scripts if you were to run them one by one. However the way we set it up was that we included all the DB stuff with running the scripts which makes more sense based on the way the whole project was built. Keeping that in mind, the addlibrraies.py file and the code you mentioned above are required for connecting the scripts with the DB. You can't run filldb.py without both the DB present and addlibraries.py present (which needs to connect to the DB to update it hence that code).

Also another thing that might add to this conversation is that the way DB was set up since before I started working on this project, was that filldb.py only "fills" the db, as in the "spots" it has to fill with data have to be already present, filldb.py doesn't create new spots to fill. So assume the popularity metric was run in August and now you have all the popularity data for august. Now, if you just run filldb.py, nothing would change in the db because there are no empty slots to fill for the month of august, you would have to first run addlibraries whicch creates an empty dated slot for each of the libraries, and then now if you run filldb, the new data would be filled into the db.

[relhajj]

So in conclusion, based on the way we set up the separation between website related code and scripts/DB related code, yes that code above would be necessary for filling the DB (but not for the website itself).

[snadi]

Because of the reliance on the django models for writing data to the common DB, we need the librarycomparison/models.py. I moved all django related setup to addlibraries.py to keep it one place (WIP in PR #24 ) but eventually, we should find a way to keep the models common between the scripts and the website