PII Catcher for Files and Databases

Overview

PIICatcher is a data catalog and scanner for PII and PHI information. It finds PII data in your databases and file systems and tracks critical data. The data catalog can be used as a foundation to build governance, compliance and security applications.

Check out AWS Glue & Lake Formation Privilege Analyzer for an example of how piicatcher is used in production.

Quick Start

PIICatcher is available as a docker image or command-line application.

Docker

docker run tokern/piicatcher:latest db -c '/db/sqlqb'

╭─────────────┬─────────────┬─────────────┬─────────────╮
│   schema    │    table    │   column    │   has_pii   │
├─────────────┼─────────────┼─────────────┼─────────────┤
│        main │    full_pii │           a │           1 │
│        main │    full_pii │           b │           1 │
│        main │      no_pii │           a │           0 │
│        main │      no_pii │           b │           0 │
│        main │ partial_pii │           a │           1 │
│        main │ partial_pii │           b │           0 │
╰─────────────┴─────────────┴─────────────┴─────────────╯

Command-line

To install use pip:

python3 -m venv .env
source .env/bin/activate
pip install piicatcher

# Install Spacy English package
python -m spacy download en_core_web_sm

# run piicatcher on a sqlite db and print report to console
piicatcher db -c '/db/sqlqb'
╭─────────────┬─────────────┬─────────────┬─────────────╮
│   schema    │    table    │   column    │   has_pii   │
├─────────────┼─────────────┼─────────────┼─────────────┤
│        main │    full_pii │           a │           1 │
│        main │    full_pii │           b │           1 │
│        main │      no_pii │           a │           0 │
│        main │      no_pii │           b │           0 │
│        main │ partial_pii │           a │           1 │
│        main │ partial_pii │           b │           0 │
╰─────────────┴─────────────┴─────────────┴─────────────╯

API

from piicatcher import scan_file_object, scan_database

pii_types = scan_file_object(...)
catalog = scan_database(...)

Supported Technologies

PIICatcher supports the following filesystems:

POSIX
AWS S3 (for files that are part of tables in AWS Glue and AWS Athena)
Google Cloud Storage (Coming Soon)
ADLS (Coming Soon)

PIICatcher supports the following databases:

Sqlite3 v3.24.0 or greater
MySQL 5.6 or greater
PostgreSQL 9.4 or greater
AWS Redshift
Oracle
AWS Glue/AWS Athena
Snowflake

Documentation

For advanced usage refer documentation PIICatcher Documentation.

Survey

Please take this survey if you are a user or considering using PIICatcher. The responses will help to prioritize improvements to the project.

Contributing

For Contribution guidelines, PIICatcher Developer documentation.

dm03514/piicatcher