/ATD

Task for the Advanced Topics in Databases course - DWS MSc Spring '22

Primary LanguagePython

ATD

Task for the ATD course - DWS MSc Spring 2022

ATD

TODO

  • Create a crawler to get articles and save them in csv files
  • Add them to Postgres
  • Connect Postgres with Python using a connector (psycopg3)
  • Read credentials from config file
  • Add create directory if not exists in extract_body.py
  • Fix article_path.csv
  • Add threshold to relevant docs in text_query.py
  • Add columns to show in text_query.py
  • Show lines that have keywords (grep maybe?)
  • Add requirements.txt
  • Fix - In text_query.py:301 -> check if list empty
  • Move links.csv to csv_files
  • Add show vector in text_query.py output
  • Use GIN index on docvec column
  • Displaying docvec troublesome in terminal
  • Add comments