How would you automate this process so that we can get new datasets every day? I would create a telegram bot Everytime New article is posted It works.
What file format would you use to store this data? I used csv format, I think sql is better
How would you evaluate the quality of the collected data? I think It is useless. Because I couldn't clean the data well and understand completely