tasos-py/Search-Engines-Scraper

Showing zero results on bing

MShirazAhmad opened this issue · 6 comments

My Code:

from search_engines import Bing
engine = Bing()
results = engine.search("lums")
links = results.links()
print(links)
# Searching Bing                                                                 
# []  

I am getting links with zero length, whatever I search.

Yes, I'm getting no results too. They probably made some updates in their HTML/API. Thanks for letting me know, I'll fix it as soon as possible.

Actually, they didn't make any updates, they just stopped supporting our User-Agent. So, all I had to do was change the FAKE_USER_AGENT string and now it works.

Is there any way to change FAKE_USER_AGENT in the loop without changing it directly from the source code?

What loop do you mean? If you want to change the User-Agent per request (although I don't see why you would do that) you could probably do it by overriding the SearchEngine._get_page() method. If you want to change the User-Agent of an engine, use .set_headers(). Is this what you mean?

Yes, that's what I was looking for. Thanks

Each time I run, I add 1 to the number variable, which resolved the zero results issue permanently.

number=number+1
engine.set_headers({'User-Agent':f"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/{number}.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/{number}.36"})