tb push error during tinybird setup
NirbhaySirsikar opened this issue · 8 comments
When the tb push
command is ran, I get the following error in the terminal:
** Running 'dub_lead_events_pipe'
Error:
** Failed running ./pipes\dub_lead_events_pipe.pipe:
** Invalid results, read description below to fix the error:
** Failed pushing pipe dub_lead_events_pipe__checker: [Error] Missing columns: 'ip' 'qr' while processing query: 'SELECT timestamp, event_id, click_id, link_id, customer_id, event_name, metadata, url, country, city, device, browser, os, referer, qr, ip FROM nirbhay_new_dub_workspace.dub_lead_events AS dub_lead_events', required columns: 'link_id' 'timestamp' 'event_name' 'customer_id' 'click_id' 'country' 'city' 'qr' 'device' 'event_id' 'os' 'ip' 'referer' 'metadata' 'url' 'browser', maybe you meant: 'link_id', 'timestamp', 'event_name', 'customer_id', 'click_id', 'country', 'city', 'device', 'event_id', 'os', 'referer', 'metadata', 'url' or 'browser'. (UNKNOWN_IDENTIFIER)
@NirbhaySirsikar can you share dub_click_events_pipe.pipe
file that is in packages folder
dub_click_events_pipe.pipe
NODE mv
SQL >
SELECT
timestamp,
click_id,
link_id,
url,
country,
city,
device,
browser,
os,
referer,
qr,
ip
FROM dub_click_events
TYPE materialized
DATASOURCE dub_click_events_mv
@NirbhaySirsikar,
In the datasources folder, there are two files: dub_click_events_mv.datasource and dub_click_events.datasource. Please check both of these files to see if they contain the qr and ip fields.
Yes, they do have ip and pr fields, and I haven’t modified any files in the /packages
directory. I did a git pull
a few days ago, and ever since then, I’ve had this error. Before that, everything was working fine.
sorry my bad I asked you to check in a wrong file, can you please check these in dub_lead_events_mv.datasource
& dub_lead_events.datasource
or can you share these file contents
dub_lead_events_pipe.pipe
NODE mv
SQL >
SELECT
timestamp,
event_id,
click_id,
link_id,
customer_id,
event_name,
metadata,
url,
country,
city,
device,
browser,
os,
referer,
qr,
ip
FROM dub_lead_events
TYPE materialized
DATASOURCE dub_lead_events_mv
dub_lead_events.datasource
SCHEMA >
`timestamp` DateTime64(3) `json:$.timestamp` DEFAULT now(),
`event_id` String `json:$.event_id`,
`event_name` String `json:$.event_name`,
`metadata` String `json:$.metadata`,
`customer_id` String `json:$.customer_id`,
`click_id` String `json:$.click_id`,
`link_id` String `json:$.link_id`,
`url` String `json:$.url`,
`country` LowCardinality(String) `json:$.country`,
`city` String `json:$.city`,
`region` String `json:$.region`,
`latitude` String `json:$.latitude`,
`longitude` String `json:$.longitude`,
`device` LowCardinality(String) `json:$.device`,
`device_model` LowCardinality(String) `json:$.device_model`,
`device_vendor` LowCardinality(String) `json:$.device_vendor`,
`browser` LowCardinality(String) `json:$.browser`,
`browser_version` String `json:$.browser_version`,
`os` LowCardinality(String) `json:$.os`,
`os_version` String `json:$.os_version`,
`engine` LowCardinality(String) `json:$.engine`,
`engine_version` String `json:$.engine_version`,
`cpu_architecture` LowCardinality(String) `json:$.cpu_architecture`,
`ua` String `json:$.ua`,
`bot` UInt8 `json:$.bot`,
`referer` String `json:$.referer`,
`referer_url` String `json:$.referer_url`,
`ip` String `json:$.ip`,
`qr` UInt8 `json:$.qr`
ENGINE "MergeTree"
ENGINE_PARTITION_KEY "toYYYYMM(timestamp)"
ENGINE_SORTING_KEY "timestamp, link_id, customer_id"
dub_lead_events_mv.datasource
SCHEMA >
`timestamp` DateTime64(3),
`event_id` String,
`click_id` String,
`link_id` String,
`customer_id` String,
`event_name` LowCardinality(String),
`metadata` String,
`url` String,
`country` LowCardinality(String),
`city` String,
`device` LowCardinality(String),
`browser` LowCardinality(String),
`os` LowCardinality(String),
`referer` String,
`qr` UInt8,
`ip` String
ENGINE "MergeTree"
ENGINE_PARTITION_KEY "toYYYYMM(timestamp)"
ENGINE_SORTING_KEY "link_id, timestamp"
Hey! This should be fixed now – we had an outdated Tinybird schema/pipes before, but it was updated recently – sorry about that!
Feel free to open a new issue if you're still running into issues!