Cannot flatten schema due to schema map being a tuple[OrderedDict, list]

[khorabi55]

Following the readme tutorial, I reproduced the steps:

schema_map = generator.deduce_schema(
    input_data=table_data
)


schema = generator.flatten_schema(schema_map)

the input_data is a dictionary, which I specified in the generator configurations.

The exception raised is: Exception: Unexpected type '<class 'tuple'>' for schema_map

[bxparks]

Can you post the input_data that you are using, so that I can reproduce the error?

[khorabi55]

I can't, sorry. It's company data, but I can inform that is a json response that I've sliced up like this, so it turned as a simple dictionary:

table_data = data_json[0]["data_report02"]

[bxparks]

You are prudent to avoid posting internal company data. The usual way around that is to reduce your input data to the smallest needed to reproduce the bug (you should need only a single line), then replace the data elements with dummy data, while preserving the structure of the input.

I cannot see what your input data format is, but clearly it is not consistent with what SchemaGenerator expects. I admit that the documentation for the input_format parameter and the dict option is not very good. These features grew organically, with contributions from multiple people, so the docs was not up-to-date. The only good way to figure that out was to trace the source code, which is not user-friendly. So I just submitted some additional documentation which hopefully makes is very clear what the expected input format is. I also added a unit test as an example. Please take a look at revised section in the README.md: https://github.com/bxparks/bigquery-schema-generator#UsingAsLibrary

I hope this is sufficient, because I am not able to support a for-profit company with my free labor much further. Good luck!

[khorabi55]

Thanks for the effort on documentation! I went for google cloud storage since having trouble, but I will go back to this issue, and BigQuery, in short notice and perhaps use your generator again.

[bxparks]

Ok, good luck.
I'm moving this to the newly created Disccusions since this is not a bug with bigquery-schema-generator.