Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic when trying to process a large input file #65

Open
xtian opened this issue Jun 27, 2019 · 3 comments
Open

Panic when trying to process a large input file #65

xtian opened this issue Jun 27, 2019 · 3 comments

Comments

@xtian
Copy link

xtian commented Jun 27, 2019

I'm seeing a panic when I try to process my input file (4.7M lines). It processes ok at around 25k, though:

~/src/statemap$ target/release/statemap sessions2.out > out.svg
thread 'main' panicked at 'already borrowed: BorrowMutError', src/libcore/result.rs:997:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
~/src/statemap$ head -n 25000 sessions2.out > test.out
~/src/statemap$ target/release/statemap test.out > out.svg
test.out: 24999 records processed, 23674 rectangles
~/src/statemap$ head -n 30000 sessions2.out > test.out
~/src/statemap$ target/release/statemap test.out > out.svg
thread 'main' panicked at 'already borrowed: BorrowMutError', src/libcore/result.rs:997:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
~/src/statemap$ RUST_BACKTRACE=1 cargo run -- test.out > out.svg
    Finished dev [unoptimized + debuginfo] target(s) in 0.21s
     Running `target/debug/statemap test.out`
thread 'main' panicked at 'attempt to subtract with overflow', src/statemap.rs:491:51
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:71
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:59
             at src/libstd/panicking.rs:197
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:211
   4: <std::panicking::begin_panic::PanicPayload<A> as core::panic::BoxMeUp>::get
             at src/libstd/panicking.rs:474
   5: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:381
   6: std::panicking::try::do_call
             at src/libstd/panicking.rs:308
   7: <T as core::any::Any>::type_id
             at src/libcore/panicking.rs:85
   8: <T as core::any::Any>::type_id
             at src/libcore/panicking.rs:49
   9: statemap::statemap::StatemapEntity::subsume
             at src/statemap.rs:491
  10: statemap::statemap::Statemap::trim
             at src/statemap.rs:1045
  11: statemap::statemap::Statemap::ingest
             at src/statemap.rs:1554
  12: statemap::main
             at src/main.rs:293
  13: std::rt::lang_start::{{closure}}
             at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libstd/rt.rs:64
  14: std::panicking::try::do_call
             at src/libstd/rt.rs:49
             at src/libstd/panicking.rs:293
  15: panic_unwind::dwarf::eh::read_encoded_pointer
             at src/libpanic_unwind/lib.rs:87
  16: <std::panicking::begin_panic::PanicPayload<A> as core::panic::BoxMeUp>::get
             at src/libstd/panicking.rs:272
             at src/libstd/panic.rs:388
             at src/libstd/rt.rs:48
  17: std::rt::lang_start
             at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libstd/rt.rs:64
  18: statemap::main
@xtian
Copy link
Author

xtian commented Jun 27, 2019

Ah, just noticed attempt to subtract with overflow. I must have some funky data in there.

@bcantrill
Copy link
Contributor

Would you mind pointing me to the data? Clearly, the failure mode should be cleaner than this. ;)

@xtian
Copy link
Author

xtian commented Jun 28, 2019

Sure thing: test.out

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants