polarsignals/frostdb

update frostdb to last version, aggregating queries with time causes panic

jicanghaixb opened this issue · 4 comments

Hi, today i update frostdb to last version, I use sql query, aggregating queries with time causes panic!

select max(value), min(value),count(value) where sample_type="request_duration" group by name
+------+------------+------------+--------------+
| NAME | MAX(VALUE) | MIN(VALUE) | COUNT(VALUE) |
+------+------------+------------+--------------+
| ott | 1008419.00 | 0.00 | 600000 |
+------+------------+------------+--------------+
4 columns, 1 rows in set

Elapsed: 141.736175ms

select max(value), min(value),count(value) where sample_type="request_duration" group by name, second(5)
Elapsed: 141.006832ms
ERR: rpc error: code = Unknown desc = runtime error: index out of range [0] with length 0

I found table.go ,see below:
case rg, ok := <-rowGroups:
if !ok {
r := converter.NewRecord() //panic

The final panic location is below:
func (b *OptInt64Builder) NewArray() arrow.Array {
dataAsBytes := unsafe.Slice((*byte)(unsafe.Pointer(&b.data[0])), len(b.data)*arrow.Int64SizeBytes) //panic

Looks like this happens when the data is empty. I think we've never run into this as we don't have int64 columns that are ever empty.

@thorfour can you have a look at this?

By the way @jicanghaixb, I'm curious, what are you building with frostdb? We'd love to know!

@brancz currently, we use prometheus and thanos for our monitoring, but they cannot solve the problem of high cardinality telemetry data analysis. Therefore, we want to use frostdb as an auxiliary tool to solve this problem.

I rollback version to November 29th, the same data, time aggregation query is not a problem.

Thanks @jicanghaixb I'm taking a look at this!