.Map returning None
gpopovic opened this issue · 2 comments
gpopovic commented
Sometimes my .Map() function returns None..
Instead of skipping this value, None will be added to the collection and later i'll get errors when None tries to be saved to bigquery.
Currently i'm fixing this by using .FlatMap() and returning [] instead of None and [Value] when I have a valid value.
Is this valid behavior of .Map or is it a bug?
chamikaramj commented
I believe it is correct to add 'None' to the PCollection if your Map function returns 'None' values.
BigQuery sync failing for 'None' values is a bug that we recently fixed in Apache Beam repository.
apache/beam#765
gpopovic commented
@chamikaramj makes sense, thanks