karolow/instatools

A Python package to facilitate acquisition and preprocessing of data from Instagram API.

instatools

A Python package to facilitate acquisition and preprocessing of data from Instagram API. Unlike other similar tools, it is equipped with features to mitigate problems arising from API instability, e.g. exception handling and easy completion of missing data. It was designed for city researchers who usually outsource similar tasks to external developers.

Features

• Extracting posts and images from the unofficial Instagram API via hashtag and location endpoints.
• Automated parsing and preprocessing of data returned by the API.
• Easy data categorisation based on selected hashtags.

Installation

Download the package:

git clone https://github.com/karolow/instatools.git
cd instatools

and install it locally using pip:

pip install -e .

Usage

from instatools.scraping import HashtagScraper, LocationScraper, Scraper
from instatools.preprocessing import HashtagPosts, LocationPosts

1. Pick the project name/directory.

project_name = 'medialab_katowice'

2. Get a session id from your browser (you must be logged in to Instagram).

session_id = 'your_session_id'

3. You can scrape Instagram API via two endpoints:

• Hashtag (#medialabkatowice)

h = HashtagScraper(session_id, project_name)
h.set_api_session()
h.extract_posts('https://www.instagram.com/explore/tags/medialabkatowice/')

• Location (Medialab Katowice)

l = LocationScraper(session_id, project_name)
l.set_api_session()
l.extract_posts('https://www.instagram.com/explore/locations/379985438715032/medialab-katowice/')

Obtained json files will be stored automatically in the following directories:

└── posts
    ├── hashtag
    │   ├── 1.json
    │   ├── 2.json
    │   ├── 3.json
    │   └── (...)
    └── location
        ├── 1.json
        ├── 2.json
        ├── 3.json
        └── (...)

4. Load data from JSON files.

h = HashtagPosts.from_json_files('posts/hashtag')
l = LocationPosts.from_json_files('posts/location')

5. You can add up extracted posts to create one larger collection.

p = h + l

6. While working with Instagram data, you might want to remove junk content by specifying posts with unwanted hashtags.

p.remove_posts(['#spam', '#ads'])

You can also store these hashtags in the txt file.

p.remove_posts(file_path='junk_hashtags.thx')

In which case they should be separated with commas.

#spam, #ads, #promotion

7. Access posts

p.posts

or convert to Pandas data frame

p.to_df()
p.df

8. You can assign posts to categories based on hashtags. Store them in a CSV file.

category,hashtags
Shared Cities,#sccm2020
Workshop,"#workshop, #arduino"

and run

p.set_custom_categories(file_path='custom_categories.csv', name='categories')

9. Basic analysis

p.popular_categories(category='categories', pct=True)
p.popular_hashtags(n=10, pct=False)