gojekfarm/beast

Support Recursive protobuf data types

Closed this issue · 0 comments

For a recursive protobuf schema like below:

message TestRecursiveMessage {
    string string_value = 1;
    TestRecursiveMessage recursive_message = 2;
}

The Bigquery schema generation logic fails with StackOverflow because of the Recursive data type. Since Recursive data types are supported in protobuf, the recursive protobuf should be converted to bigquery schema with a limit on the depth of the schema.

Bigquery supports up to 15 levels deep nesting of bigquery fields. Thus 15 can be set as limit on the depth of the schema.