range_partitioning does not work
halil-burak opened this issue · 4 comments
When we try to create our BQ tables, we want to use range_partitioning in one of them. We could not generate the range partition on of them. I've been using this module for a while but not able to use range_partitioning on a table. There is no error but the range partitioning is not applied, is not shown on plan output.
We are using this terraform registry and Google's provider for this.
https://registry.terraform.io/modules/terraform-google-modules/bigquery/google/latest
Can you share your Terraform config?
Hi,
Of course. I'm sharing the scripts and configs below.
module usage with dataset and table with range partitioning
module "testing_the_range_partitioning" {
source = "terraform-google-modules/bigquery/google"
version = "~> 4.4"
dataset_id = "testing_the_range_partitioning"
dataset_name = "test range"
description = "Dataset containing tables for person and job staging data"
project_id = var.project_id
location = "EU"
tables = [
{
table_id = "testing_the_range_partitioning_table",
schema = "./schema.json",
time_partitioning = null,
deletion_protection = false,
range_partitioning = {
field = "personId",
range = {
start = "1",
end = "1000000",
interval = "10000",
},
},
expiration_time = null,
clustering = null,
labels = {},
},
]
access = [] # pass empty not to conflict with below
}
provider.tf for google provider --> Google as a provider
provider "google" {
version = "~> 3.90"
project = var.project_id
region = var.region
credentials = file("key.json")
}
schema.json file
[
{
"mode": "NULLABLE",
"name": "personId",
"type": "INTEGER"
},
{
"fields": [
{
"name": "actor",
"type": "STRING"
},
{
"name": "action",
"type": "STRING"
}
],
"mode": "REPEATED",
"name": "personData",
"type": "RECORD"
},
{
"mode": "NULLABLE",
"name": "createdAt",
"type": "TIMESTAMP"
},
{
"mode": "NULLABLE",
"name": "updatedAt",
"type": "TIMESTAMP"
}
]
I've figured out what was the issue. We were using version 4.3 and 4.4 of this module, I upgraded to version 5.X and started getting schema errors. After some research, I found out that I had to use file() function for the json schema. Then it worked.
@halil-burak Glad it worked for you!