SteamDataHarvester is a multi-threaded Python tool for efficiently scraping Steam user data. It navigates through user networks, collects game data, and maintains a unique tracking system to avoid duplicate processing. This tool is ideal for analyzing trends and social networks within the Steam gaming community.
- Multi-threaded data scraping for optimized performance
- Collects game playtime data from user profiles
- Navigates through friends' networks to gather extended data
- Avoids duplicate processing with a smart tracking system
Before you begin, ensure you have met the following requirements:
- Python 3.x installed
requestsandpandaslibraries installed- A Steam API key
Clone the repository to your local machine:
git clone https://github.com/hpekkan/SteamDataHarvester.gitSteamDataHarvester is a multi-threaded Python tool for efficiently scraping Steam user data. It navigates through user networks, collects game data, and maintains a unique tracking system to avoid duplicate processing. This tool is ideal for analyzing trends and social networks within the Steam gaming community.
- Multi-threaded data scraping for optimized performance
- Collects game playtime data from user profiles
- Navigates through friends' networks to gather extended data
- Avoids duplicate processing with a smart tracking system
Before you begin, ensure you have met the following requirements:
- Python 3.x installed
- 'requests' and 'pandas' libraries installed
- A Steam API key
Clone the repository to your local machine:
git clone https://github.com/hpekkan/SteamDataHarvester.gitTo use SteamDataHarvester, follow these steps:
- Add your Steam API key and desired initial SteamID to the script.
- Run
python scrapper.pyin your terminal. - Data will be saved in the 'user_data' folder.
- Run
python merger.pyin your terminal. - Merged csv file be saved in './{counter}_user_data.csv'.
To contribute, follow these steps:
- Fork this repository.
- Create a branch:
git checkout -b <branch_name> - Make your changes and commit them:
git commit -m '<commit_message> - Push to the original branch:
git push origin <project_name>/<location> - Create the pull request.
Alternatively, see the GitHub documentation on creating a pull request.
If you want to contact me, you can reach me at huseyinpekkan1163@gmail.com.
This project uses the following license: Apache License 2.0.