Use scraping result directly in python

[so3500]

Please kindly help me with my issue.
i'm testing GoogleScraper/Examples/basic.py and successfully got results.

this is my config info.

config = {
    'use_own_ip': True,
    'keyword': 'how to make blabla',
    'search_engines': ['google'],
    'num_pages_for_keyword': 1,
    'scrape_method': 'selenium',
    'sel_browser': 'chrome',
    'do_caching': False,
}

and i checked results in shell.

figure 1

And I can only use the data in Python as shown below.

figure 2

I know that the results from figure 1 are stored in the database
I also know that there is a way to add 'output_filename' field to config and save it to a file and then use it to call it.

config = {
    'use_own_ip': True,
    'keyword': 'how to make blabla',
    'search_engines': ['google'],
    'num_pages_for_keyword': 1,
    'scrape_method': 'selenium',
    'sel_browser': 'chrome',
    'do_caching': False,
    'output_filename': 'output.csv',
}

But,

I want to use the results from figure 1 directly in Python code (title, link etc)

Any Idea??

[alon001]

Give me a few days, and I will show you where in the code you could greb it and print or what ever you need. I did it, but busy next days.

[alon001]

add the below lines in database.py file, set_values_from_parser:
print("PARSED LINK IS: ", link['link'])
print("PARSED TITLE IS: ", link['title'])
print("PARSED SNIPPET IS: ", link['snippet'])
add it after the lines:
Link(
link=link['link'],
snippet=link['snippet'],
title=link['title'],
visible_link=link['visible_link'],
domain=parsed.netloc,
rank=link['rank'],
serp=self,
link_type=key,
)
You can add similarly to print to log:
logger.info("PARSED LINK IS: ")
logger.info(link['link'])
etc.
I don't remember, but if needed for the logger print, add at the top of database.py file the next import (if it is missing I think it is needed to enable logger print):
import logging
logger = logging.getLogger(name)
I guess you can now add it to any specific log file you want, in the same way.

[so3500]

I really appreciate your kind and quick reply.
I will test the answer you have uploaded as soon as possible, upload the results, and close the issue.

[Sinyii]

search = sqlalchemy_session.query(ScraperSearch).all()[-1]
for serp in search.serps:
    for link in serp.links:
        print("KW:%s" %(serp.query))
        print(link.snippet)

You can change snippet to the element you want to use. :)