audienceproject/spark-dynamodb

IllegalArgumentException: No region provided

Opened this issue · 1 comments

Hi there,
I'm trying to upgrade to the latest version:
spark-dynamodb_2.12-1.1.0
but no matter what I do I get the above error: "IllegalArgumentException: No region provided"

The code works on this version:
spark-dynamodb_2.11-1.0.4

Code here:

df = spark.read.schema(source_raw_schema).format("dynamodb") \
  .option("region", "eu-west-1") \
  .option("roleArn", "arn:aws:iam::xxx") \
  .option("tableName", "yyy") \
  .load()

Can you help?

Thanks!

Hi - not related to your issue and I apologize for that. I was just wondering if you could help me out to understand how to load data from dynamo using the AP library? Can you provide me with a code snippet on how to connect dynamo to spark and then load the data to a df? Thanks