`InvalidArchive("Invalid zip header")` when attempting to read zip archive on linux

Question

`InvalidArchive("Invalid zip header")` when attempting to read zip archive on linux

TheMatrixMaster opened this issue 2 years ago · 2 comments

Description

I may be missing something obvious here, but I am getting this invalid archive error when attempting to read a zip archive on my linux machine. When I run the same code but compiled on my m1 macbook though it works fine (the file was moved to the macbook using ftp so maybe that changes the headers or something?). Does anyone have any idea why this could be? Ideally, I'd like to stick with this crate to handle my zip files on the linux machine.

Source code

let zipfile = std::fs::File::open(main_path).expect("Failed to read file using io");
let mut archive = zip::ZipArchive::new(zipfile).expect("Failed to open zip archive using zip crate");

Machine

Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit

OS

NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"

Zip file type

HONGQU20230620.zip: Zip archive data, at least v2.0 to extract

Backtrace

Processing new data from zip file: "/home/ec2-user/hq/fw_test/.watchdir/HONGQU20230620.zip"
thread 'main' panicked at 'Failed to open zip archive using zip crate: InvalidArchive("Invalid zip header")', src/main.rs:68:34
stack backtrace:
[root@ip-172-31-80-39 fw_test]#    0:     0x55d6ed1a135a - std::backtrace_rs::backtrace::libunwind::trace::h9a6b80bbf328ba5d
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:     0x55d6ed1a135a - std::backtrace_rs::backtrace::trace_unsynchronized::hd162ec543a11886b
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x55d6ed1a135a - std::sys_common::backtrace::_print_fmt::h78a5099be12f51a6
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/sys_common/backtrace.rs:65:5
   3:     0x55d6ed1a135a - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::ha1c5390454d74f71
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/sys_common/backtrace.rs:44:22
   4:     0x55d6ed1bcfbf - core::fmt::write::h9ffde816c577717b
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/fmt/mod.rs:1254:17
   5:     0x55d6ed19e925 - std::io::Write::write_fmt::h88186074961638e4
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/io/mod.rs:1698:15
   6:     0x55d6ed1a1125 - std::sys_common::backtrace::_print::h184198273ed08d59
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/sys_common/backtrace.rs:47:5
   7:     0x55d6ed1a1125 - std::sys_common::backtrace::print::h1b4d8e7add699453
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/sys_common/backtrace.rs:34:9
   8:     0x55d6ed1a27ce - std::panicking::default_hook::{{closure}}::h393bcea75423915a
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:269:22
   9:     0x55d6ed1a2575 - std::panicking::default_hook::h48c64f31d8b3fd03
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:288:9
  10:     0x55d6ed1a2d2e - std::panicking::rust_panic_with_hook::hafdc493a79370062
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:691:13
  11:     0x55d6ed1a2c29 - std::panicking::begin_panic_handler::{{closure}}::h0a64bc82e36bedc7
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:582:13
  12:     0x55d6ed1a17c6 - std::sys_common::backtrace::__rust_end_short_backtrace::hc203444fb7416a16
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/sys_common/backtrace.rs:150:18
  13:     0x55d6ed1a2982 - rust_begin_unwind
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:578:5
  14:     0x55d6ed01e413 - core::panicking::panic_fmt::h0f6ef0178afce4f2
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/panicking.rs:67:14
  15:     0x55d6ed01e8f3 - core::result::unwrap_failed::h8090202169109f9c
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/result.rs:1687:5
  16:     0x55d6ed031328 - core::result::Result<T,E>::expect::hf29ffaa2e7fba51c
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/result.rs:1046:23
  17:     0x55d6ed02ac11 - fw_test::watch::he3e3715fbeea7664
                               at /home/ec2-user/hq/fw_test/src/main.rs:67:47
  18:     0x55d6ed02044b - fw_test::main::h20bdbecf3775b423
                               at /home/ec2-user/hq/fw_test/src/main.rs:120:21
  19:     0x55d6ed02149b - core::ops::function::FnOnce::call_once::h5deaf7e971aa7742
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/ops/function.rs:250:5
  20:     0x55d6ed02293e - std::sys_common::backtrace::__rust_begin_short_backtrace::h63e1ce7839c5aab9
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/sys_common/backtrace.rs:134:18
  21:     0x55d6ed020ed1 - std::rt::lang_start::{{closure}}::h23a70f1e6946b9c5
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/rt.rs:166:18
  22:     0x55d6ed19af7e - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::hb1327dc2ef3fecdf
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/ops/function.rs:287:13
  23:     0x55d6ed19af7e - std::panicking::try::do_call::h4044173225fe83dd
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:485:40
  24:     0x55d6ed19af7e - std::panicking::try::hd8a722c09d156a53
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:449:19
  25:     0x55d6ed19af7e - std::panic::catch_unwind::hd2ca07971cf0119b
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panic.rs:140:14
  26:     0x55d6ed19af7e - std::rt::lang_start_internal::{{closure}}::h26d89d595cf47b70
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/rt.rs:148:48
  27:     0x55d6ed19af7e - std::panicking::try::do_call::hf47aa1aa005e5f1a
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:485:40
  28:     0x55d6ed19af7e - std::panicking::try::h73d246b2423eaf4e
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panicking.rs:449:19
  29:     0x55d6ed19af7e - std::panic::catch_unwind::hbaaeae8f1b2f9915
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/panic.rs:140:14
  30:     0x55d6ed19af7e - std::rt::lang_start_internal::h76f3e81e6b8f13f9
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/rt.rs:148:20
  31:     0x55d6ed020eaa - std::rt::lang_start::h58d06e4d3334f08a
                               at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/std/src/rt.rs:165:17
  32:     0x55d6ed02062e - main
  33:     0x7fa1cd0dc13a - __libc_start_main
  34:     0x55d6ed01eada - _start
  35:                0x0 - <unknown>

[1]+  Exit 101                ./target/debug/fw_test .watchdir/

Answer 1 · 2023-06-25T01:17:44.000Z

If you can, put up the file and take a look？

Answer 2 · 2023-06-27T07:31:57.000Z

I figured out what was wrong:

The filewatcher was picking up the zip archive before the FTP job had finished, so I was attempting to unzip a partial / corrupted zip that hadn't finished downloading. To temporarily fix the issue, I am sleeping the thread for some time before attempting to unzip the archive.

Ideally, one should use the first solution described in this (stack overflow post)[https://stackoverflow.com/questions/1211384/verify-whether-ftp-is-complete-or-not], but since I don't have control over the sending ftp protocol, I'll implement an busy wait while loop that periodically checks if the zip transfer has finished with some exponential backoff and put all that computation in a new thread.