Streaming Data Example doesn't change values
farkhad opened this issue · 2 comments
farkhad commented
Tried this example for streaming data into ClickHouse https://github.com/ClickHouse/ch-go/blob/main/examples/insert/main.go
with changed values for body, name
...
// Append 10 rows to initial data block.
for i := 0; i < 10; i++ {
body.AppendBytes([]byte("Hello"))
ts.Append(now)
name.Append("name")
sevText.Append("INFO")
sevNumber.Append(10)
arr.Append([]string{"foo", "bar", "baz"})
}
// Insert single data block.
input := proto.Input{
{Name: "ts", Data: ts},
{Name: "severity_text", Data: &sevText},
{Name: "severity_number", Data: sevNumber},
{Name: "body", Data: body},
{Name: "name", Data: name},
{Name: "arr", Data: arr},
}
if err := conn.Do(ctx, ch.Query{
Body: "INSERT INTO test_table_insert VALUES",
// Or "INSERT INTO test_table_insert (ts, severity_text, severity_number, body, name, arr) VALUES"
Input: input,
}); err != nil {
panic(err)
}
// Stream data to ClickHouse server in multiple data blocks.
var blocks int
if err := conn.Do(ctx, ch.Query{
Body: input.Into("test_table_insert"), // helper that generates INSERT INTO query with all columns
Input: input,
// OnInput is called to prepare Input data before encoding and sending
// to ClickHouse server.
OnInput: func(ctx context.Context) error {
...
// Append new values:
for i := 0; i < 10; i++ {
body.AppendBytes([]byte("Hello2"))
ts.Append(now)
name.Append("name2")
sevText.Append("DEBUG")
sevNumber.Append(11)
arr.Append([]string{"foo", "bar", "baz"})
}
...
Why severity_text changed to DEBUG, but severity_number, body, name are the same as in initial data block?
SELECT *
FROM test_table_insert
Query id: e59eeb35-3942-42b0-90d0-2f905025cdf3
┌────────────────────────────ts─┬─severity_text─┬─severity_number─┬─body──┬─name─┬─arr─────────────────┐
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
└───────────────────────────────┴───────────────┴─────────────────┴───────┴──────┴─────────────────────┘
┌────────────────────────────ts─┬─severity_text─┬─severity_number─┬─body──┬─name─┬─arr─────────────────┐
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ INFO │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
│ 2010-01-01 13:22:33.000345678 │ DEBUG │ 10 │ Hello │ name │ ['foo','bar','baz'] │
└───────────────────────────────┴───────────────┴─────────────────┴───────┴──────┴─────────────────────┘
120 rows in set. Elapsed: 0.002 sec.
ernado commented
I think it is because example should be following:
input := proto.Input{
{Name: "ts", Data: &ts},
{Name: "severity_text", Data: &sevText},
{Name: "severity_number", Data: &sevNumber},
{Name: "body", Data: &body},
{Name: "name", Data: &name},
{Name: "arr", Data: &arr},
}
References should be used so Reset() can be called by input on all coumns.
Thank you, I'll fix it soon in examples.