audienceproject/spark-dynamodb

Question on throughput parameter

Opened this issue · 0 comments

So far this code is working great for me , thank you very much.

I am setting throughput as 25000 , but getting throttled error, and Dynamodb is hovering around 40000 WCU in metrics tab and sometimes it gets to 55000 WCU. Is there any other parameter that throughput depends on?

final.repartition(300).write.option("tableName", table).option("writeBatchSize",25).option("throughput",25000)

com.amazonaws.services.dynamodbv2.model.RequestLimitExceededException: Throughput exceeds the current throughput limit for your account. Please contact AWS Support at https://aws.amazon.com/support request a limit increase (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: RequestLimitExceeded; Request ID: L5VFK8QJA1E2J5GG58S26K1VGRVV4KQNSO5AEMVJF66Q9ASUAAJG)