using Overwrite flag on save injest the key only
shadibch opened this issue · 4 comments
shadibch commented
We tried to use the following code:
aggregations.filter("id is not null").groupBy("id").agg(collect_set("imsi")). write
.format("org.apache.spark.sql.redis")
.option("table", "tacs")
.mode(SaveMode.Overwrite)
.option("ttl", 300)
.option("key.column", "id")
.save()
This saves the key only. We should use the Append mode flag to save the dataframe
fe2s commented
Hi @shadibch ,
could you please the sample dataframe (after the transformation) that you try to save?
shadibch commented
Hello
The problem is not related with the dataframe
Once we use the tag SaveMode.Overwrite) in any dataframe to be saved into redis it saves only the key.
fe2s commented
Could you provide some example I can reproduce please?
sazzad16 commented
Closed due to inactivity.