In python following readme example but unable to print out any data
ahsan-n opened this issue · 3 comments
version: spark-dynamodb_2.12:1.1.0
dynamoDf = spark.read.option("tableName", "mytable").option("region","eu-west-1") \ .format("dynamodb") \ .load() dynamoDf.show(100)
this gives me empty table
++ || ++ || || ++
but when i use load().count() it showed me the right count of total entries in table.
yeah, it happened to me as well, trying passing the schema.
@ahsan-n can you tell how you are importing the module? Getting error while importing. Was able to download it as mentioned in readme.
I have the same error as above, get empty table in return
Have given the schema and got it working
from pyspark.sql.types import *
fields = [StructField("email", StringType(), True)
, StructField("first_name", StringType(), True)
, StructField("last_name", StringType(), True)]
schema = StructType(fields)
# Load data from dynamo into spark dataframe
dynamoDf = spark.read.option("tableName","db_dev_users") \
.option("region",region) \
.schema(schema) \
.format("dynamodb") \
.load()
Have been using this in the databricks environment, and installed the packages from Maven repo
version 1.1.2, and it works
Thanks @kaifee