Conflict occurred creating export directory already exists
ashay10001 opened this issue · 1 comments
ashay10001 commented
when i write two bigquery dataframes in for loop using function saveAsBigQueryTable(projectid+schemaname+tablename) it gives the error when writing the table in bigquery where it gives error Conflict occurred creating export directory. Path gs://mybucket_name/hadoop/tmp/spark-bigquery/spark-bigquery-1595825912980=521003067 already exists
plz reply with solution
samelamin commented
hmmm that sounds like a race condition issue? can you validate if that is the case? try sleeping between writing frames for a certain amount of time
Id like to know more if its due to a race condition or perhaps the way you are saving the table