Extraction of Data from public API's
-
Install Python dependencies
pip install requirements.txt
-
Run postman.py
python postman.py
or
python3 postman.py
This would extract the data and you can access it through your mysql shell by executing commands:
- First you have to take it into sql mode so:
/sql
- Connect it to your database using:
Use root@localhost and enter your sql password
/connect
- Use Postman
- Access the data by using the following command:
select * from allData where category = [category you wish to see]
What things you need to install the software and how to install them
- Python
- mysql
A step by step series of examples that tell you how to get a development env running
Run postman.py and you will start to see data getting extracted.
* addData
* Category
* API
* Description
* Auth
* HTTPS
* CORS
* Link
Attaching postman database for demo
- Followed concept of OOPS
- Support for handling authentication requirements & token expiration of server
- Support for pagination to get all data
- Develop work around for rate limited server
- Crawled all API entries for all categories and stored it in a mysql database
- May provide With multiple types of Database Support like Mongodb, Postgres.
- May Deploy it online for demonstration purpose.
- Can add more features like gui support with flask.
- Can add docker for single command application.
- Aish Kanodia