tasos-py/Search-Engines-Scraper

How to stop the search

stuartathompson opened this issue · 3 comments

Sorry for posting a rudimentary question but there is no real instruction or documentation.

After putting in the example it keeps searching Google non-stop. How am I supposed to interrupt this or set other options?

engine = Google()
results = engine.search("my query")
links = results.links()

It's not mentioned in readme, but there are options to limit or interrupt .search(). To limit the number of results, simply set the .search(pages=) argument to a lower number. For example, the code below will request only the first two pages,

    results = engine.search("my query", 2)

If you want to stop a running .search(), you could use a keyboard interrupt signal (CTRL+C). KeyboardInterrupt exceptions are caught internally in .search(), and they cause it to stop making requests and return the results.

Also, in other-versions you'll find an async version of this project, that should allow you to tun it in the background.

Thanks! Is this documented anywhere? I'd happily do a pull request with more details on the available options. I'm not a good enough coder to wade into the source to find out. I'm working on a project that now will use this extensively (I hope) so happy to help.

No, there is no documentation, besides what's in README. If you have the time to write something, you're welcome to contribute. A good place to start is the parent class, "/search_engines/engine.py". Its public methods have short docstrings