Stress test rustfix

[killercup]

Can we fix 10k suggestions in a reasonable time?

[killercup]

So, I wrote killercup@9565a44 and ran it with env RUST_LOG=cargo_fix=trace cargo test --release -- fix_perf --nocapture.

Output

   Compiling cargo-fix v0.2.0 (file:///Users/pascal/Projekte/rustfix/cargo-fix)
    Finished release [optimized] target(s) in 7.00 secs
     Running /Users/pascal/Projekte/rustfix/target/release/deps/all-e0c9c7f0d97e176d

running 1 test

···················································
running "cargo-fix" "fix"
dur: 9.532ms
exit: exit code: 0
stderr ---
DEBUG 2018-05-07T10:53:29Z: cargo_fix: invoking cargo-fix as cargo subcommand
    Checking foo v0.1.0 (file:///Users/pascal/Projekte/rustfix/target/generated-tests/test0)
DEBUG 2018-05-07T10:53:29Z: cargo_fix: invoking cargo-fix as rustc wrapper
TRACE 2018-05-07T10:53:29Z: cargo_fix: cargo-fix as rustc got file Some("src/lib.rs")
TRACE 2018-05-07T10:53:29Z: cargo_fix: start rustfixing "src/lib.rs"
TRACE 2018-05-07T10:53:38Z: cargo_fix: got 198308842 bytes of output from rustc, will now collect suggestions
DEBUG 2018-05-07T10:53:38Z: cargo_fix: collected 1000 suggestions for `src/lib.rs`
DEBUG 2018-05-07T10:53:38Z: cargo_fix: applying 1000 fixes to src/lib.rs
      Fixing src/lib.rs (1000 fixes)
    Finished dev [unoptimized + debuginfo] target(s) in 9.49 secs

···················································
fix_10k_warnings took Duration { secs: 9, nanos: 532221569 }
test perf::fix_perf ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 17 filtered out

(ignore the "fix_10k_warnings" name -- fixing 10k warnings took too long)

Results

• rustc takes about 9 seconds to compile 1000 single-line functions
  • it generates 200MB of JSON output!
• it takes rustfix basically no time to replace this

[alexcrichton]

Nice! I'd also be surprised if rustfix was the slow part of anything in the pipeline here, but rather executing rustc three times per crate is probably the slow part...

[killercup]

Yeah, actual rustfixing should be fast, but I was surprised that buffering the compiler's output could actually have a perf impact (we could in theory switch to a buffered reader). A 200MB string is probably at the very extreme end of the potential spectrum of real-life rustfix usage.

[ehuss]

I'm going to close since I don't think we've had too much of an issue with performance.