unexpected warning ： shard.rs:106: Invalid batch: found batch start inside batch
i18nsite opened this issue · 6 comments
i18nsite commented
I'm writing a command line application. When my program exits, I use keyspace.persist(FlushMode::SyncAll)?;
I not close the database (is there a shutdown function?)
The following warn occasionally appears at 2 times start my cli. ( video https://www.loom.com/share/c1c1a71d06fb495bad85bacda73b219e )
Is there a way to block this warning?
WARN fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.3/src/journal/shard.rs:106: Invalid batch: found batch start inside batch
WARN fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.3/src/journal/shard.rs:70: Truncating shard to 88
WARN fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.3/src/journal/shard.rs:213: Invalid batch: missing terminator, but last batch, so probably incomplete, discarding to keep atomicity
WARN fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.3/src/journal/shard.rs:70: Truncating shard to 88
WARN fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.3/src/journal/shard.rs:106: Invalid batch: found batch start inside batch
WARN fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.3/src/journal/shard.rs:70: Truncating shard to 0
WARN fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.3/src/journal/shard.rs:213: Invalid batch: missing terminator, but last batch, so probably incomplete, discarding to keep atomicity
WARN fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.3/src/journal/shard.rs:70: Truncating shard to 0
WARN fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.3/src/journal/shard.rs:106: Invalid batch: found batch start inside batch
WARN fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.3/src/journal/shard.rs:70: Truncating shard to 0
WARN fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.3/src/journal/shard.rs:213: Invalid batch: missing terminator, but last batch, so probably incomplete, discarding to keep atomicity
WARN fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.3/src/journal/shard.rs:70: Truncating shard to 0
WARN fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.3/src/journal/shard.rs:106: Invalid batch: found batch start inside batch
WARN fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.3/src/journal/shard.rs:70: Truncating shard to 0
WARN fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.3/src/journal/shard.rs:213: Invalid batch: missing terminator, but last batch, so probably incomplete, discarding to keep atomicity
WARN fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.3/src/journal/shard.rs:70: Truncating shard to 0
marvin-j97 commented
When my program exits, I use keyspace.persist(FlushMode::SyncAll)?;
On Drop, flush is called automatically https://github.com/fjall-rs/fjall/blob/main/src/keyspace.rs#L80
I not close the database (is there a shutdown function?)
Dropping is the shutdown function. You can check trace logs to see if the keyspace is properly dropping in your application.
Is there a way to block this warning?
I may need to adjust the verbosity of some of the logging. Depending on the logger you use, you may be able to set a filter though.
marvin-j97 commented
I have adjusted the logging levels to be debug and trace in the next version
i18nsite commented
more log
❯ ./i18/test.sh
+ '[' -z '' ']'
++ realpath /Users/z/i18n/rust/i18
+ PROJECT=/Users/z/i18n/rust/i18/tests/yml
+ direnv exec . cargo run -- -d /Users/z/i18n/rust/i18/tests/yml
direnv: loading ~/i18n/rust/i18/.envrc
direnv: loading ~/i18n/rust/.envrc
Compiling i18 v0.1.203 (/Users/z/i18n/rust/i18)
Finished `dev` profile [optimized + debuginfo] target(s) in 1.57s
Running `/tmp/rust/target/debug/i18 -d /Users/z/i18n/rust/i18/tests/yml`
INFO fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:271: Opening keyspace at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache"
INFO fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:573: Creating keyspace at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache"
INFO fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:299: Spawning 11 compaction threads
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::partition: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/partition/mod.rs:121: Creating partition md
DEBUG lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:59: Opening LSM-tree at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/partitions/md"
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::partition: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/partition/mod.rs:121: Creating partition yml
DEBUG lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:59: Opening LSM-tree at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/partitions/yml"
README.md ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── 0% ⠁ 00:00:00 ETA 0s DEBUG reqwest::connect: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/reqwest-0.12.4/src/connect.rs:497: starting new connection: http://38.242.220.222:8850/
README.md ==================================================================================================================================================== 100% 00:00:00 ETA 0s DEBUG fjall::flush::worker: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/flush/worker.rs:113: flush worker: write locking flush manager
DEBUG fjall::flush::worker: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/flush/worker.rs:121: flush worker: No tasks collected
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:660: fsync thread: exiting because tree is dropping
✅ i18n.site translate
❯ ./i18/test.sh
+ '[' -z '' ']'
++ realpath /Users/z/i18n/rust/i18
+ PROJECT=/Users/z/i18n/rust/i18/tests/yml
+ direnv exec . cargo run -- -d /Users/z/i18n/rust/i18/tests/yml
direnv: loading ~/i18n/rust/i18/.envrc
direnv: loading ~/i18n/rust/.envrc
Compiling i18 v0.1.203 (/Users/z/i18n/rust/i18)
Finished `dev` profile [optimized + debuginfo] target(s) in 1.63s
Running `/tmp/rust/target/debug/i18 -d /Users/z/i18n/rust/i18/tests/yml`
INFO fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:271: Opening keyspace at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache"
INFO fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:516: Recovering keyspace at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache"
INFO fjall::journal: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/journal/mod.rs:64: Recovering journal from "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/journals/0"
DEBUG fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/journal/shard.rs:106: Invalid batch: found batch start inside batch
DEBUG fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/journal/shard.rs:213: Invalid batch: missing terminator, but last batch, so probably incomplete, discarding to keep atomicity
DEBUG fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/journal/shard.rs:106: Invalid batch: found batch start inside batch
DEBUG fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/journal/shard.rs:213: Invalid batch: missing terminator, but last batch, so probably incomplete, discarding to keep atomicity
DEBUG fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/journal/shard.rs:106: Invalid batch: found batch start inside batch
DEBUG fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/journal/shard.rs:213: Invalid batch: missing terminator, but last batch, so probably incomplete, discarding to keep atomicity
DEBUG fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/journal/shard.rs:106: Invalid batch: found batch start inside batch
DEBUG fjall::journal::shard: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/journal/shard.rs:213: Invalid batch: missing terminator, but last batch, so probably incomplete, discarding to keep atomicity
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:528: Recovered active journal at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/journals/0"
DEBUG lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:59: Opening LSM-tree at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/partitions/md"
INFO lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:798: Recovering LSM-tree at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/partitions/md"
DEBUG lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:936: Recovering disk segments from "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/partitions/md"
DEBUG lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:997: Recovered 0 segments
DEBUG fjall::recovery: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/recovery.rs:101: Keyspace seqno is now 2
DEBUG lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:59: Opening LSM-tree at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/partitions/yml"
INFO lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:798: Recovering LSM-tree at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/partitions/yml"
DEBUG lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:936: Recovering disk segments from "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/partitions/yml"
DEBUG lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:997: Recovered 0 segments
DEBUG fjall::recovery: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/recovery.rs:101: Keyspace seqno is now 2
INFO fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:299: Spawning 11 compaction threads
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::flush::worker: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/flush/worker.rs:113: flush worker: write locking flush manager
DEBUG fjall::flush::worker: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/flush/worker.rs:121: flush worker: No tasks collected
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
✅ i18n.site translate
❯ ./i18/test.sh
+ '[' -z '' ']'
++ realpath /Users/z/i18n/rust/i18
+ PROJECT=/Users/z/i18n/rust/i18/tests/yml
+ direnv exec . cargo run -- -d /Users/z/i18n/rust/i18/tests/yml
direnv: loading ~/i18n/rust/i18/.envrc
direnv: loading ~/i18n/rust/.envrc
Finished `dev` profile [optimized + debuginfo] target(s) in 0.27s
Running `/tmp/rust/target/debug/i18 -d /Users/z/i18n/rust/i18/tests/yml`
INFO fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:271: Opening keyspace at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache"
INFO fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:516: Recovering keyspace at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache"
INFO fjall::journal: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/journal/mod.rs:64: Recovering journal from "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/journals/0"
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:528: Recovered active journal at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/journals/0"
DEBUG lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:59: Opening LSM-tree at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/partitions/md"
INFO lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:798: Recovering LSM-tree at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/partitions/md"
DEBUG lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:936: Recovering disk segments from "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/partitions/md"
DEBUG lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:997: Recovered 0 segments
DEBUG fjall::recovery: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/recovery.rs:101: Keyspace seqno is now 2
DEBUG lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:59: Opening LSM-tree at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/partitions/yml"
INFO lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:798: Recovering LSM-tree at "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/partitions/yml"
DEBUG lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:936: Recovering disk segments from "/Users/z/i18n/rust/i18/tests/yml/.i18n/cache/partitions/yml"
DEBUG lsm_tree::tree: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lsm-tree-1.0.4/src/tree.rs:997: Recovered 0 segments
DEBUG fjall::recovery: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/recovery.rs:101: Keyspace seqno is now 2
INFO fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:299: Spawning 11 compaction threads
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:673: compaction: waiting for work
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::flush::worker: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/flush/worker.rs:113: flush worker: write locking flush manager
DEBUG fjall::flush::worker: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/flush/worker.rs:121: flush worker: No tasks collected
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
DEBUG fjall::keyspace: /Users/z/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fjall-1.0.4/src/keyspace.rs:679: compaction thread: exiting because tree is dropping
marvin-j97 commented
@i18n-now Looks good to me, do you expect different logs? The missing terminator, but last batch logs are to be expected, I think with V2 I will be able to get rid of them... but they don't necessarily indicate data loss.
i18nsite commented
I see , I thought it was a problem with my usage that caused the error.
If the data is automatically written when dropped, why is there still a warning?
I would expect a way to explicitly avoid this prompt, as the data being possibility correct also means there is a possibility of loss.
Losing data is quite terrifying.
Additionally, I am using fjall as a cache in my translation tool (which is still under development): https://github.com/search?q=repo%3Ai18n-site%2Frust%20fjall&type=code
The project is currently under development, and Fjall is very useful ^_^.
I hope there can be a performance monitoring comparison that can monitor the performance of each version.
Something similar to https://github.com/benchmark-action/github-action-benchmark
marvin-j97 commented
I see , I thought it was a problem with my usage that caused the error.
If the data is automatically written when dropped, why is there still a warning?
That's a bit of an oversight on my part, it's not critical, but it would be a breaking change to "fix" it. So it's gonna be a data format V2 change.
I would expect a way to explicitly avoid this prompt, as the data being possibility correct also means there is a possibility of loss.
Losing data is quite terrifying.
Me too honestly, it's not 100% transparent right now, but it's fixable.
I hope there can be a performance monitoring comparison that can monitor the performance of each version. Something similar to https://github.com/benchmark-action/github-action-benchmark
I planned on adding something like this, but it's a bit of effort to setup. I didn't know about that GH action, thanks for sharing!