tap-playstore [Google Play (GCS Export)]

tap-playstore is a Singer tap to extract Google Play Store Console Reports that are available on google cloud storage.

The following streams can be integrated using this tap:

stats_by_dimension_buyers_7d
stats_by_dimension_crashes
stats_by_dimension_gcm
stats_by_dimension_installs
stats_by_dimension_ratings
stats_by_dimension_retained_installers
stats_overview_crashes
stats_overview_gcm
stats_overview_installs
stats_overview_ratings
reviews
store_performance_country
store_performance_traffic_source
subscriptions_country
earnings
play_balance_krw
sales_reports

note: the streams prefixed with stats_by_dimension consolidate all reports by dimension for the given metric into one common stream.

Built with the Meltano Tap SDK for Singer Taps.

Project Home

https://github.com/haleemur/tap-playstore

The project's source can be viewed on github

Installation

Install from PyPi:

pipx install tap-playstore

Install from GitHub:

pipx install git+https://github.com/haleemur/tap-playstore.git@main

Capabilities

catalog
state
discover
about
stream-maps
batch

Configuration

Accepted Config Options

Setting	Required	Default	Description
service_account_json_file	False	None	Google Cloud Service Account JSON file
service_account_json_str	False	None	Google Cloud Service Account JSON string
start_date	True	None	The earliest record date to sync
bucket_name	True	None	The GCS Bucket where Play Console Reports are stored.
stream_maps	False	None	Config object for stream maps capability. For more information check out Stream Maps.
stream_map_config	False	None	User-defined config values to be used within map expressions.

NOTE: At least one of [service_account_json_file, service_account_json_str] are required.

A full list of supported settings and capabilities for this tap is available by running:

tap-playstore --about

Configure using environment variables

This Singer tap will automatically import any environment variables within the working directory's .env if the --config=ENV is provided, such that config values will be considered if a matching environment variable is set either in the terminal context or in the .env file.

Usage

You can easily run tap-playstore by itself or in a pipeline using Meltano.

Executing the Tap Directly

tap-playstore --version
tap-playstore --help
tap-playstore --config CONFIG --discover > ./catalog.json

Developer Resources

Follow these instructions to contribute to this project.

Initialize your Development Environment

pipx install poetry
poetry install
poetry run pre-commit install

Create and Run Tests

Tests require a test_config.json file to be present. Currently, running tests require an active connection to google cloud.

TODO: write proper mocks for google cloud storage.

The test_config.json file should mimic the structure of sample_config.json file.

{
    "start_date": "2024-01-01",
    "bucket_name": "pubsite_prod_rev_00000000000123456789",
    "service_account_json_file": "test_credentials.json"
}

Create tests within the tests subfolder and then run:

poetry run pytest

You can also test the tap-playstore CLI interface directly using poetry run:

poetry run tap-playstore --help

Testing with Meltano

Note: This tap will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.

Next, install Meltano (if you haven't already) and any needed plugins:

# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-playstore
meltano install

Update the meltano config (or populate the environment variables / modify your .env file)

    config:
      start_date: '2024-01-01T00:00:00Z'
      service_account_json_file: $GOOGLE_PLAY_CREDENTIALS_FILE
      bucket_name: $GOOGLE_PLAY_BUCKET

Now you can test and orchestrate using Meltano:

# Test invocation:
meltano invoke tap-playstore --version
# OR run a test `elt` pipeline:
meltano elt tap-playstore target-jsonl

SDK Dev Guide

See the dev guide for more instructions on how to use the SDK to develop your own taps and targets.