spotify/dbeam

Provide descriptions for fields

larruda opened this issue · 3 comments

Hello. I have a requirement to have descriptions in the fields of the tables on BigQuery. When loading an Avro export from DBeam into BQ, the descriptions are set according to a hard-coded sentence DBeam sets, such as:

From sqlType 4 INTEGER

I need a way to set the descriptions myself. Could you please let me know the best way to accomplish this in your option on DBeam? I can prepare a PR. Thank you.

DBeam does not support that.

One idea to implement that would be to add another command line argument pointing to an avsc schema, which would be used to export.

While that still does not happen, I recommend to process data from DBeam exports in scio or beam translating to the appropriate schema.

Agree, this is a vital feature to have in dbeam.

Can be closed?