Search results are completely different than what it should be
Opened this issue · 2 comments
Piece of code:
query = "Philippines Palawan"
for result in search(query, num_results=50):
if "wikipedia" in result: do something
else: something else
This gives me the following list:
https://www.tripadvisor.com/Tourism-g294255-Palawan_Island_Palawan_Province_Mimaropa-Vacations.html
https://www.hotels.com/go/philippines/best-palawan-things-to-do
https://guidetothephilippines.ph/articles/ultimate-guides/palawan-travel-guide
https://philippinetourismusa.com/top-destinations/palawan/
https://www.connections.be/en/tours/philippines/the-palawan-adventure
https://stock.adobe.com/search?k=%22palawan%20philippines%22
https://www.cntraveler.com/galleries/2015-07-13/visiting-the-most-beautiful-island-in-the-world-palawan-philippines
https://philippines.travel/destinations/palawan
https://en.wikivoyage.org/wiki/Palawan
https://www.travel-palawan.com/about-palawan/
https://www.lightfoottravel.com/en/asia/philippines/palawan
https://www.elnidoresorts.com/en/
https://www.instagram.com/dream_palawan/?hl=en
http://www.gis-reseau-asie.org/en/philippines-last-frontier-what-perspectives-palawan
https://pcsd.gov.ph/
https://ebird.org/region/PH-PLW
https://palawan-news.com/
https://en.unesco.org/biosphere/aspac/palawan
https://www.connections.be/en/tours/philippines/the-palawan-adventure
https://stock.adobe.com/search?k=%22palawan%20philippines%22
https://philippines.travel/destinations/palawan
https://en.wikivoyage.org/wiki/Palawan
https://www.travel-palawan.com/about-palawan/
https://www.lightfoottravel.com/en/asia/philippines/palawan
https://www.elnidoresorts.com/en/
https://www.instagram.com/dream_palawan/?hl=en
http://www.gis-reseau-asie.org/en/philippines-last-frontier-what-perspectives-palawan
https://pcsd.gov.ph/
https://ebird.org/region/PH-PLW
https://palawan-news.com/
https://en.unesco.org/biosphere/aspac/palawan
https://en.wikivoyage.org/wiki/Palawan
https://www.travel-palawan.com/about-palawan/
https://www.lightfoottravel.com/en/asia/philippines/palawan
https://www.elnidoresorts.com/en/
https://www.instagram.com/dream_palawan/?hl=en
http://www.gis-reseau-asie.org/en/philippines-last-frontier-what-perspectives-palawan
https://pcsd.gov.ph/
https://ebird.org/region/PH-PLW
https://palawan-news.com/
https://en.unesco.org/biosphere/aspac/palawan
https://ebird.org/region/PH-PLW
http://www.gis-reseau-asie.org/en/philippines-last-frontier-what-perspectives-palawan
https://palawan-news.com/
https://en.unesco.org/biosphere/aspac/palawan
https://ebird.org/region/PH-PLW
https://palawan-news.com/
https://en.unesco.org/biosphere/aspac/palawan
https://en.unesco.org/biosphere/aspac/palawan
https://en.unesco.org/biosphere/aspac/palawan
If, when I do a google search, wikipedia is the exact first result, why can't I find it in 50 results through the program?
When I search for it there are the tripadvisor and hotels.com results and a few other travel-related ones before the Wikipedia result, but I am surprised that Wikipedia does not show up at all in the results. Perhaps your location is affecting it?
The location/IP is the same, I don't use proxy or VPN. When I search normally on google, the wikipedia link is either (almost always) the first one, or a couple results down. Using the library, inexistent. I had to finish my scraping script by using Selenium which is very heavy since it simulates a real browser.