Akumuli crashing on query
sptrakesh opened this issue · 2 comments
sptrakesh commented
I am running via the latest Docker image (akumuli:0.8.80-skylake). The following query via the Grafana plugin causes the container to crash.
spt_akumuli.1.sullcoizlnqg@docker-desktop | 2020-06-05 23:02:06,816 http [INFO] Cursor 140025295594008 created
spt_akumuli.1.sullcoizlnqg@docker-desktop | 2020-06-05 23:02:06,816 http [INFO] Cursor 140025295594008 started
spt_akumuli.1.sullcoizlnqg@docker-desktop | 2020-06-05 23:02:06,817 Main [INFO] Parsing query:
spt_akumuli.1.sullcoizlnqg@docker-desktop | 2020-06-05 23:02:06,818 Main [INFO] {
spt_akumuli.1.sullcoizlnqg@docker-desktop | "select": "request.counter",
spt_akumuli.1.sullcoizlnqg@docker-desktop | "range": {
spt_akumuli.1.sullcoizlnqg@docker-desktop | "from": "20200605T230206.779",
spt_akumuli.1.sullcoizlnqg@docker-desktop | "to": "20200605T200206.779"
spt_akumuli.1.sullcoizlnqg@docker-desktop | },
spt_akumuli.1.sullcoizlnqg@docker-desktop | "where": "",
spt_akumuli.1.sullcoizlnqg@docker-desktop | "order-by": "series",
spt_akumuli.1.sullcoizlnqg@docker-desktop | "apply": "",
spt_akumuli.1.sullcoizlnqg@docker-desktop | "limit": "1000000",
spt_akumuli.1.sullcoizlnqg@docker-desktop | "pivot-by-tag": [
spt_akumuli.1.sullcoizlnqg@docker-desktop | "resource"
spt_akumuli.1.sullcoizlnqg@docker-desktop | ]
spt_akumuli.1.sullcoizlnqg@docker-desktop | }
spt_akumuli.1.sullcoizlnqg@docker-desktop |
spt_akumuli.1.sullcoizlnqg@docker-desktop | 2020-06-05 23:02:06,819 Main [ERROR] MergeIterator - broken invariant
spt_akumuli.1.sullcoizlnqg@docker-desktop | 2020-06-05 23:02:06,820 Main [ERROR] Terminating (core dumped)
Akumuli logs also show this error:
2020-06-05 23:09:30,277 Main [INFO] 18446744073709550400 Trying to open tree, repair status - OK, addr: 20537
2020-06-05 23:09:30,278 Main [ERROR] Empty rescue points list found, leaf-node data was lost
2020-06-05 23:09:30,278 Main [ERROR] Repair needed, id=1145
Lazin commented
How your tags look like? Is 'resource' the only tag in the series name?
sptrakesh commented
No, there are additional tags such as method
, resource
, ipaddress
, status
and mimeType
.
Sorry, an OS update wiped out the old logs. I was able to trace the issue back further to what looked like a database corruption (probably caused when writing the series). The query causing the crash seemed to be an after effect of that corruption.
Will create another issue if I run into it again. Until then probably only option is close this issue.