Recommended branch: Central

Description

Price indexr is intended to get a set of product prices for a given simultaneous search on Google shopping and Bing shopping and store it locally. To ensure you will get only the desired results, you can filter what will be saved locally based on the title each individual product. This program can be prompted manually, or scheduled to run by other softwares like cron, or Windows's task scheduler.

How can this help me?

This can be used to satisfy business and personal necessities, for stablishing market prices, help on calculating price indexes for specific types of products or monitoring the price of a product you want to buy.

# Requirements

Python version 3.8 or superior and packages:
- bs4
- sqlalchemy
- requests
Any popular internet browser installed, and know it's User Agent (If you want to troubleshoot)

Recommended for scheduling

For macOS/Unix operating systems:

Bash
cron/anacron or cronitor installed

For Windows operating systems:

How to use?

Currently this project is under construction, and may not be working properly (~~maybe not working at all~~)

The intended usage is on a terminal:

python price_indexr.py "<Database connection string or '.csv'>" "my product search" "<location code [optional]>"

This will use python to run the price_indexr.py script with 3 different arguments in order:

A connection string to a database supported by SQLAlchemy on it's Included Dialects, or simply ".csv" to save in a text file;
A search that you would type on google's search field. Must be inside quotes or double qutoes if it contains more than one word;
[Optional] A location code for the country (e.g. "us" for the United States, or "pt-br" for Brazil). If not included, google search engine will guess the country by the IP address that you are using.

How the search works?

The search results could come with a lot of similar products that doesn't meet your expectations. For that reason, every word you type in the search argument is a filter, Price_indexr works with two kinds of filters:

Positive: this filter contains every word that you demand to be in the title of the product. This also will be used to fill the google shopping's search bar.
Negative: this filter contains every word that you demand NOT to be in the title of the product. This will never be used to fill the google shopping's search bar, as it would bring more unwanted results.

So your search would look like: "gtx 1660 -pc -notebook"*

In the example above, "gtx" and "1660" are positive filters and "pc" and "notebook" are negative. In this case you want to search the "gtx 1660" graphics card's price, but as the results might bring unwanted computers and notebooks that comes with this graphics card equiped, you should use negative filters such as those to avoid adding them to your database. The positive filters would naturally prevent you from getting similar graphics cards such as "gtx 1650" and ensuring you get data only from the desired model.

*Note that positive filters should always come first.

Scheduling on Unix/macOS with Bash

Example of automation with crontab

```
PYTHON_PATH="<path to desired python interpreter>"
SCRIPT_PATH="<path to price_indexr.py>"
DB_CON="<your database connection string>" 
# or ".csv" to save in a text file in the same folder of your script instead of a database

crontab -e
@monthly "$PYTHON_PATH" "$SCRIPT_PATH" "$DB_CON" "my product search"
# To check the schedules made:
crontab -l
```

Scheduling on Windows with PowerShell

Example of automation with Windows Task Scheduler

```
$PYTHON_PATH = "<path to desired python interpreter>"
$SCRIPT_PATH = "<path to price_indexr.py>"
$DB_CON = "<your database connection string>" 

$price_indexr_action = New-ScheduledTaskAction 
    -Execute "python3 $SCRIPT_PATH $DB_CON 'my product search'"
$monthly = New-ScheduledTaskTrigger -Monthly -At 0:00am
$task_<unique name> = Register-ScheduledTask 
    -Action $price_indexr_action
    -Trigger $monthly 
    -TaskName "<unique name>" 
    -Description "<Your Description>"
$task_<unique name> | Set-ScheduledTask
```

If a database was selected to store the data, a table called "price_indexr-my_product_search" will be created in the database on the first execution. Next executions will append new data to the same table. If you preferred a text file, the name of the file will follow the same naming pattern.

Troubleshooting

Depending on the user-agent, the results may or not appear for Price_indexr, usually, changing the user-agent on the line 125 of the script will solve the problem.

VFLins/price_indexr