Addon does not start anymore
MariusHerget opened this issue · 1 comments
MariusHerget commented
Problem/Motivation
The database does not start probably anymore. I have updated to Hassio 4.6 today but the error consistent before as well.
Expected behavior
Normal startup.
Actual behavior
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 00-banner.sh: executing...
-----------------------------------------------------------
Add-on: InfluxDB
Scalable datastore for metrics, events, and real-time analytics
-----------------------------------------------------------
Add-on version: 4.4.1
You are running the latest version of this add-on.
System: Ubuntu 21.10 (amd64 / qemux86-64)
Home Assistant Core: 2022.4.6
Home Assistant Supervisor: 2022.04.0
-----------------------------------------------------------
Please, share the above information when looking for help
or support in, e.g., GitHub, forums or the Discord chat.
-----------------------------------------------------------
[cont-init.d] 00-banner.sh: exited 0.
[cont-init.d] 01-log-level.sh: executing...
Log level is set to INFO
[cont-init.d] 01-log-level.sh: exited 0.
[cont-init.d] create-users.sh: executing...
[cont-init.d] create-users.sh: exited 0.
[cont-init.d] influxdb.sh: executing...
[cont-init.d] influxdb.sh: exited 0.
[cont-init.d] kapacitor.sh: executing...
[cont-init.d] kapacitor.sh: exited 0.
[cont-init.d] nginx.sh: executing...
[cont-init.d] nginx.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[16:09:05] INFO: Chronograf is waiting until InfluxDB is available...
[16:09:05] INFO: Kapacitor is waiting until InfluxDB is available...
[16:09:06] INFO: Starting the InfluxDB...
unexpected fault address 0x7f3eb00c508f
fatal error: fault
[signal SIGBUS: bus error code=0x2 addr=0x7f3eb00c508f pc=0x12a436f]
goroutine 100 [running]:
runtime.throw(0x169e54a, 0x5)
/usr/local/go/src/runtime/panic.go:774 +0x72 fp=0xc000ede7d0 sp=0xc000ede7a0 pc=0x431272
runtime.sigpanic()
/usr/local/go/src/runtime/signal_unix.go:391 +0x455 fp=0xc000ede800 sp=0xc000ede7d0 pc=0x446925
encoding/binary.bigEndian.Uint16(...)
/usr/local/go/src/encoding/binary/binary.go:101
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*indirectIndex).UnmarshalBinary(0xc000119830, 0x7f3eb00a3f83, 0x3556c, 0x35574, 0x0, 0x0)
/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/reader.go:1236 +0x1cf fp=0xc000ede8b8 sp=0xc000ede800 pc=0x12a436f
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*mmapAccessor).init(0xc000fb0d20, 0x0, 0x0, 0x0)
/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/reader.go:1359 +0x2a2 fp=0xc000ede960 sp=0xc000ede8b8 pc=0x12a4d82
github.com/influxdata/influxdb/tsdb/engine/tsm1.NewTSMReader(0xc0000ccc60, 0xc000edeaa0, 0x1, 0x1, 0x3, 0x3, 0x0)
/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/reader.go:239 +0x190 fp=0xc000ede9d8 sp=0xc000ede960 pc=0x129e7c0
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*FileStore).Open.func1(0xc000982750, 0xc0000a2240, 0x0, 0xc0000ccc60)
/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/file_store.go:543 +0x125 fp=0xc000edefc0 sp=0xc000ede9d8 pc=0x12c36d5
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0xc000edefc8 sp=0xc000edefc0 pc=0x460f61
created by github.com/influxdata/influxdb/tsdb/engine/tsm1.(*FileStore).Open
/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/file_store.go:535 +0x444
goroutine 1 [chan receive]:
github.com/influxdata/influxdb/tsdb.(*Store).loadShards(0xc0004aa200, 0x0, 0x0)
/go/src/github.com/influxdata/influxdb/tsdb/store.go:421 +0xaae
github.com/influxdata/influxdb/tsdb.(*Store).Open(0xc0004aa200, 0x0, 0x0)
/go/src/github.com/influxdata/influxdb/tsdb/store.go:221 +0x289
github.com/influxdata/influxdb/cmd/influxd/run.(*Server).Open(0xc00048e280, 0xc0003afd40, 0xc00048e280)
/go/src/github.com/influxdata/influxdb/cmd/influxd/run/server.go:444 +0xddd
github.com/influxdata/influxdb/cmd/influxd/run.(*Command).Run(0xc0003dadc0, 0xc0000b21b0, 0x0, 0x0, 0x0, 0xc0000b21b0)
/go/src/github.com/influxdata/influxdb/cmd/influxd/run/command.go:149 +0xde0
main.(*Main).Run(0xc0003aff20, 0xc0000b21b0, 0x0, 0x0, 0xc0003aff20, 0x43f0ca)
/go/src/github.com/influxdata/influxdb/cmd/influxd/main.go:81 +0x147
main.main()
/go/src/github.com/influxdata/influxdb/cmd/influxd/main.go:45 +0x13d
goroutine 19 [syscall]:
os/signal.signal_recv(0x0)
/usr/local/go/src/runtime/sigqueue.go:147 +0x9c
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:23 +0x22
created by os/signal.init.0
/usr/local/go/src/os/signal/signal_unix.go:29 +0x41
goroutine 35 [select]:
github.com/influxdata/influxdb/vendor/go.opencensus.io/stats/view.(*worker).start(0xc0001fcaa0)
/go/src/github.com/influxdata/influxdb/vendor/go.opencensus.io/stats/view/worker.go:154 +0x100
created by github.com/influxdata/influxdb/vendor/go.opencensus.io/stats/view.init.0
/go/src/github.com/influxdata/influxdb/vendor/go.opencensus.io/stats/view/worker.go:32 +0x57
goroutine 36 [IO wait]:
internal/poll.runtime_pollWait(0x7f3eb02902e0, 0x72, 0x0)
/usr/local/go/src/runtime/netpoll.go:184 +0x55
internal/poll.(*pollDesc).wait(0xc000227298, 0x72, 0x0, 0x0, 0x16a0d8c)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Accept(0xc000227280, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_unix.go:384 +0x1f8
net.(*netFD).accept(0xc000227280, 0x0, 0x0, 0x0)
/usr/local/go/src/net/fd_unix.go:238 +0x42
net.(*TCPListener).accept(0xc0004a28c0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/tcpsock_posix.go:139 +0x32
net.(*TCPListener).Accept(0xc0004a28c0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/tcpsock.go:261 +0x47
github.com/influxdata/influxdb/tcp.(*Mux).Serve(0xc0004b6480, 0x259d300, 0xc0004a28c0, 0xc0004a28c0, 0x0)
/go/src/github.com/influxdata/influxdb/tcp/mux.go:75 +0x92
created by github.com/influxdata/influxdb/cmd/influxd/run.(*Server).Open
/go/src/github.com/influxdata/influxdb/cmd/influxd/run/server.go:395 +0x280
goroutine 7 [chan send]:
github.com/influxdata/influxdb/pkg/limiter.Fixed.Take(...)
/go/src/github.com/influxdata/influxdb/pkg/limiter/fixed.go:40
github.com/influxdata/influxdb/tsdb.(*Store).loadShards.func1(0xc0005b0e40, 0xc0004aa200, 0xc0000daba0, 0xc0005b0ea0, 0x168ef40, 0xc0003dc090, 0xc0006f8960, 0xc00075c4d4, 0x9, 0xc0007b0c7e, ...)
/go/src/github.com/influxdata/influxdb/tsdb/store.go:363 +0x50
created by github.com/influxdata/influxdb/tsdb.(*Store).loadShards
/go/src/github.com/influxdata/influxdb/tsdb/store.go:362 +0x1c84
goroutine 11 [chan send]:
github.com/influxdata/influxdb/pkg/limiter.Fixed.Take(...)
/go/src/github.com/influxdata/influxdb/pkg/limiter/fixed.go:40
github.com/influxdata/influxdb/tsdb.(*Store).loadShards.func1(0xc0005b0e40, 0xc0004aa200, 0xc0000daba0, 0xc0005b0ea0, 0x168ef40, 0xc0003dc090, 0xc0006f8960, 0xc00075c4d4, 0x9, 0xc0007b0c7e, ...)
/go/src/github.com/influxdata/influxdb/tsdb/store.go:363 +0x50
created by github.com/influxdata/influxdb/tsdb.(*Store).loadShards
/go/src/github.com/influxdata/influxdb/tsdb/store.go:362 +0x1c84
goroutine 13 [chan receive]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*FileStore).Open(0xc000982750, 0x0, 0x0)
/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/file_store.go:571 +0x4c1
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open(0xc00048e140, 0xc000432f00, 0xc000173520)
/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:749 +0x28d
github.com/influxdata/influxdb/tsdb.(*Shard).Open.func1(0xc0008c58c0, 0x0, 0x0)
/go/src/github.com/influxdata/influxdb/tsdb/shard.go:344 +0x3f9
github.com/influxdata/influxdb/tsdb.(*Shard).Open(0xc0008c58c0, 0xc000432fc0, 0x28)
/go/src/github.com/influxdata/influxdb/tsdb/shard.go:355 +0x2f
github.com/influxdata/influxdb/tsdb.(*Store).loadShards.func1(0xc0005b0e40, 0xc0004aa200, 0xc0000daba0, 0xc0005b0ea0, 0x168ef40, 0xc0003dc090, 0xc0006f8960, 0xc00075c4d4, 0x9, 0xc0007b0c7e, ...)
/go/src/github.com/influxdata/influxdb/tsdb/store.go:404 +0x853
created by github.com/influxdata/influxdb/tsdb.(*Store).loadShards
/go/src/github.com/influxdata/influxdb/tsdb/store.go:362 +0x1c84
goroutine 14 [runnable]:
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*partition).write(0xc0000c1bc0, 0xc00098bea0, 0xd9, 0xe0, 0xc000085220, 0x1, 0x1, 0x0, 0x0, 0x0)
/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/ring.go:260 +0x1f9
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*ring).write(0xc0000c1880, 0xc00098bea0, 0xd9, 0xe0, 0xc000085220, 0x1, 0x1, 0x2a600, 0x0, 0x0)
/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/ring.go:100 +0xb8
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Cache).WriteMulti(0xc0001d6d10, 0xc000999da0, 0x0, 0x0)
/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/cache.go:343 +0x2c3
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*CacheLoader).Load.func1(0xc000db2e00, 0xc0000c0440, 0xc000a80df0, 0xc0001d6d10, 0x0, 0x0)
/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/cache.go:747 +0x68e
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*CacheLoader).Load(0xc0000c0440, 0xc0001d6d10, 0x1, 0x1)
/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/cache.go:758 +0x97
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).reloadCache(0xc00020c280, 0x0, 0x0)
/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:2408 +0x2f1
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).Open(0xc00020c280, 0xc000432f00, 0xc0006b4000)
/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:754 +0x306
github.com/influxdata/influxdb/tsdb.(*Shard).Open.func1(0xc000740000, 0x0, 0x0)
/go/src/github.com/influxdata/influxdb/tsdb/shard.go:344 +0x3f9
github.com/influxdata/influxdb/tsdb.(*Shard).Open(0xc000740000, 0xc000432fc0, 0x28)
/go/src/github.com/influxdata/influxdb/tsdb/shard.go:355 +0x2f
github.com/influxdata/influxdb/tsdb.(*Store).loadShards.func1(0xc0005b0e40, 0xc0004aa200, 0xc0000daba0, 0xc0005b0ea0, 0x168ef40, 0xc0003dc090, 0xc0006f8960, 0xc00075c4d4, 0x9, 0xc0007b0c7e, ...)
/go/src/github.com/influxdata/influxdb/tsdb/store.go:404 +0x853
created by github.com/influxdata/influxdb/tsdb.(*Store).loadShards
/go/src/github.com/influxdata/influxdb/tsdb/store.go:362 +0x1c84
goroutine 15 [runnable]:
os.underlyingErrorIs(0x2572260, 0xc000999dd0, 0x256d7a0, 0xc0000b20e0, 0x2572260)
/usr/local/go/src/os/error.go:110 +0x1dd
os.IsNotExist(...)
/usr/local/go/src/os/error.go:93
github.com/influxdata/influxdb/tsdb.(*Store).loadShards.func1(0xc0005b0e40, 0xc0004aa200, 0xc0000daba0, 0xc0005b0ea0, 0x168ef40, 0xc0003dc120, 0xc0006f8d70, 0xc00053fba4, 0xd, 0xc0007b1612, ...)
/go/src/github.com/influxdata/influxdb/tsdb/store.go:392 +0x760
created by github.com/influxdata/influxdb/tsdb.(*Store).loadShards
/go/src/github.com/influxdata/influxdb/tsdb/store.go:362 +0x1c84
goroutine 16 [chan send]:
github.com/influxdata/influxdb/pkg/limiter.Fixed.Take(...)
/go/src/github.com/influxdata/influxdb/pkg/limiter/fixed.go:40
github.com/influxdata/influxdb/tsdb.(*Store).loadShards.func1(0xc0005b0e40, 0xc0004aa200, 0xc0000daba0, 0xc0005b0ea0, 0x168ef40, 0xc0003dc120, 0xc0006f8d70, 0xc00053fba4, 0xd, 0xc0007b1612, ...)
/go/src/github.com/influxdata/influxdb/tsdb/store.go:363 +0x50
created by github.com/influxdata/influxdb/tsdb.(*Store).loadShards
/go/src/github.com/influxdata/influxdb/tsdb/store.go:362 +0x1c84
goroutine 66 [syscall]:
syscall.Syscall(0x1, 0x1d, 0xc000184b40, 0x20b, 0xc000dced48, 0x80000000000, 0x43001a)
/usr/local/go/src/syscall/asm_linux_amd64.s:18 +0x5
syscall.write(0x1d, 0xc000184b40, 0x20b, 0x20b, 0x0, 0x0, 0x0)
/usr/local/go/src/syscall/zsyscall_linux_amd64.go:1005 +0x5a
syscall.Write(...)
/usr/local/go/src/syscall/syscall_unix.go:202
internal/poll.(*FD).Write(0xc0000db080, 0xc000184b40, 0x20b, 0x20b, 0x0, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_unix.go:268 +0x1a1
os.(*File).write(...)
/usr/local/go/src/os/file_unix.go:276
os.(*File).Write(0xc000a56018, 0xc000184b40, 0x20b, 0x20b, 0x4, 0x0, 0x0)
/usr/local/go/src/os/file.go:153 +0x77
github.com/influxdata/influxdb/tsdb.(*MeasurementFieldSet).Save(0xc0009b7ac0, 0x0, 0x0)
/go/src/github.com/influxdata/influxdb/tsdb/shard.go:1831 +0x26b
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).LoadMetadataIndex(0xc0000dc3c0, 0x8, 0x25cd540, 0xc0000b9860, 0x2b, 0xc000045290)
/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:881 +0x2cb
github.com/influxdata/influxdb/tsdb.(*Shard).Open.func1(0xc000348480, 0x0, 0x0)
/go/src/github.com/influxdata/influxdb/tsdb/shard.go:349 +0x452
github.com/influxdata/influxdb/tsdb.(*Shard).Open(0xc000348480, 0xc000432fc0, 0x2b)
/go/src/github.com/influxdata/influxdb/tsdb/shard.go:355 +0x2f
github.com/influxdata/influxdb/tsdb.(*Store).loadShards.func1(0xc0005b0e40, 0xc0004aa200, 0xc0000daba0, 0xc0005b0ea0, 0x168ef40, 0xc0003dc120, 0xc0006f8d70, 0xc00053fba4, 0xd, 0xc0007b1612, ...)
/go/src/github.com/influxdata/influxdb/tsdb/store.go:404 +0x853
created by github.com/influxdata/influxdb/tsdb.(*Store).loadShards
/go/src/github.com/influxdata/influxdb/tsdb/store.go:362 +0x1c84
[cont-finish.d] executing container finish scripts...
[cont-finish.d] 99-message.sh: executing...
[cont-finish.d] 99-message.sh: exited 0.
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.
Steps to reproduce
I am not sure how to reproduce it.
EDIT #1: Included full log.
MariusHerget commented
Deleting the internal database /usr/share/hassio/addons/data/xxxxxxx_influxdb/influxdb/influxdb/data/_internal resolved the issue. Seems like an index there was faulty.