SQLException: Zero date value prohibited

[jmriego]

Hi! First of all thank you for this project. I think it's filling really well the need of migrating data now to GCS now that so many companies are trying to move to the cloud.

I found this issue when replicating a table fro MySql: SQLException: Zero date value prohibited

I understand that Avro wouldn't allow for such a date the way MySql does but there must be a way of replicating this data. Maybe allowing to override the query for specific columns? Some command line option like this is one potential way of being able to replicate this table:
--override-query-column=activity_date:LEAST(activity_date, '1970-01-01')

Would this be possible to add to dbeam? Thanks!

[labianchin]

Can you provide some example on how you are using DBeam. I think I still don't understand why do we have some '1970-01-01' here.

[jmriego]

We are trying to use it to replicate tables from a MySQL database and it works in most cases. This issue is when we are trying to replicate a table that has dates with this value: 0000-00-00.

I was thinking of possibilities and maybe we could have some options when using dbeam to either change that zero date to some accepted range (that's why I said 1970) or maybe convert to NULL all dates that don't fall into the accepted range

[rulle-io]

@jmriego I would say it should be much more easier to replace some dates in your own SQL query.
Have you seen an option:

--sqlFile: a path to a file containing a SQL query (used instead a generated query based on table parameter)

[jmriego]

I totally missed that parameter. That definitely works for me. Thanks!