Closed this issue 7 years ago · 1 comments
After scraping multiple pages, some products have some of the same composite key, consider adding them to a set vs a list prior to insertion;
Hash the products to a set then back to a list before inserting them into the db.