TritonDataCenter/statemap

Panic when trying to process a large input file

Opened this issue · 3 comments

xtian commented

I'm seeing a panic when I try to process my input file (4.7M lines). It processes ok at around 25k, though:

~/src/statemap$ target/release/statemap sessions2.out > out.svg
thread 'main' panicked at 'already borrowed: BorrowMutError', src/libcore/result.rs:997:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
~/src/statemap$ head -n 25000 sessions2.out > test.out
~/src/statemap$ target/release/statemap test.out > out.svg
test.out: 24999 records processed, 23674 rectangles
~/src/statemap$ head -n 30000 sessions2.out > test.out
~/src/statemap$ target/release/statemap test.out > out.svg
thread 'main' panicked at 'already borrowed: BorrowMutError', src/libcore/result.rs:997:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
~/src/statemap$ RUST_BACKTRACE=1 cargo run -- test.out > out.svg
    Finished dev [unoptimized + debuginfo] target(s) in 0.21s
     Running `target/debug/statemap test.out`
thread 'main' panicked at 'attempt to subtract with overflow', src/statemap.rs:491:51
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:71
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:59
             at src/libstd/panicking.rs:197
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:211
   4: <std::panicking::begin_panic::PanicPayload<A> as core::panic::BoxMeUp>::get
             at src/libstd/panicking.rs:474
   5: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:381
   6: std::panicking::try::do_call
             at src/libstd/panicking.rs:308
   7: <T as core::any::Any>::type_id
             at src/libcore/panicking.rs:85
   8: <T as core::any::Any>::type_id
             at src/libcore/panicking.rs:49
   9: statemap::statemap::StatemapEntity::subsume
             at src/statemap.rs:491
  10: statemap::statemap::Statemap::trim
             at src/statemap.rs:1045
  11: statemap::statemap::Statemap::ingest
             at src/statemap.rs:1554
  12: statemap::main
             at src/main.rs:293
  13: std::rt::lang_start::{{closure}}
             at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libstd/rt.rs:64
  14: std::panicking::try::do_call
             at src/libstd/rt.rs:49
             at src/libstd/panicking.rs:293
  15: panic_unwind::dwarf::eh::read_encoded_pointer
             at src/libpanic_unwind/lib.rs:87
  16: <std::panicking::begin_panic::PanicPayload<A> as core::panic::BoxMeUp>::get
             at src/libstd/panicking.rs:272
             at src/libstd/panic.rs:388
             at src/libstd/rt.rs:48
  17: std::rt::lang_start
             at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libstd/rt.rs:64
  18: statemap::main
xtian commented

Ah, just noticed attempt to subtract with overflow. I must have some funky data in there.

Would you mind pointing me to the data? Clearly, the failure mode should be cleaner than this. ;)

xtian commented

Sure thing: test.out