quiltdata/quilt

cross-compatibility

Closed this issue · 6 comments

Adapt test_spec to:

  1. Create packages of different sizes: 0 MiB | 5 MiB (S, M) | 8 MiB (S, M, V) | 8 MiB + 1
  2. Using different checksum types: monolithic | chunked
  3. Compare with reference packages from the Catalog
  • S: Single-part upload
  • M: Multi-part upload
  • V: Variant multi-part upload (non-canonical, e.g. 3MiB part sizes)

Requires access to: quilt+s3://udp-spec#package=spec/parquet@f3085fd3355ca8ff22d4caac56abb4dc783258dc3910670f96d571bea664099e

Tests  
B: Browse work with manifests (with | without) chunksums (local | s3)
V: Verify work with manifests (with | without) chunksums (local | s3)
U: Uploaded package creation with upload
E: Existing package creation from existing data (valid etag | sha256 | other)
O: Optimized upload optimisation (w/KMS | wo/KMS)
  object not re-uploaded if quilt3, S3 objects have checksums
  e.g. check LastModified

Namespace created packages by quilt3 version.

Proposed test matrix
Screenshot 2024-07-24 at 17 42 21

Test files (on s3://udp-spec)
spec-setup.csv

Already done