AWS Data Wrangler

Pandas on AWS

An AWS Professional Service open source initiative | aws-proserve-opensource@amazon.com

Source	Downloads	Installation Command
PyPi		`pip install awswrangler`
Conda		`conda install -c conda-forge awswrangler`

Quick Start
Read The Docs
Community Resources
Who uses AWS Data Wrangler?
Amazon SageMaker Data Wrangler?

Quick Start

Installation command: pip install awswrangler

import awswrangler as wr
import pandas as pd

df = pd.DataFrame({"id": [1, 2], "value": ["foo", "boo"]})

# Storing data on Data Lake
wr.s3.to_parquet(
    df=df,
    path="s3://bucket/dataset/",
    dataset=True,
    database="my_db",
    table="my_table"
)

# Retrieving the data directly from Amazon S3
df = wr.s3.read_parquet("s3://bucket/dataset/", dataset=True)

# Retrieving the data from Amazon Athena
df = wr.athena.read_sql_query("SELECT * FROM my_table", database="my_db")

# Get a Redshift connection from Glue Catalog and retrieving data from Redshift Spectrum
con = wr.redshift.connect("my-glue-connection")
df = wr.redshift.read_sql_query("SELECT * FROM external_schema.my_table", con=con)
con.close()

Read The Docs

Community Resources

Please send a Pull Request with your resource reference and @githubhandle.

Who uses AWS Data Wrangler?

Knowing which companies are using this library is important to help prioritize the project internally.

Please send a Pull Request with your company name and @githubhandle if you may.

Amazon SageMaker Data Wrangler?

Amazon SageMaker Data Wrangler is a new SageMaker Studio feature that has a similar name but has a different purpose than the AWS Data Wrangler open source project.

AWS Data Wrangler is open source, runs anywhere, and is focused on code.
Amazon SageMaker Data Wrangler is specific for the SageMaker Studio environment and is focused on a visual interface.