10 Commits

Author SHA1 Message Date
Stanko K.R. ddc6ce020e Conditionally disable peer verification for ZIP streaming 2026-03-16 17:57:26 +01:00
Jeremy Daer 953942694f Handle MissingEOCD as invalid zip file, not job failure (#2572)
ZipKit::FileReader raises MissingEOCD for truncated, corrupted, or
non-zip files. This exception is a direct StandardError subclass, not
a subclass of InvalidStructure or ReadError, so it escaped as an
unhandled job failure instead of being caught and surfaced to the user.

Broaden the rescue in ZipFile::Reader#initialize to catch ReadError
(parent of InvalidStructure), MissingEOCD, and UnsupportedFeature.
2026-02-18 15:18:56 -08:00
Stanko K.R. ffce51a42f Ignore faulty ZIP files 2026-02-03 14:04:54 +01:00
Stanko K.R. 4b70c1cc3b Replace custom IO object with a transport manager IO 2026-02-02 19:31:18 +01:00
Stanko K.R. 77d9331d90 Add missing method to IO object 2026-02-02 19:06:37 +01:00
Stanko K.R. ac7d0308ac Add size to remote io 2026-02-02 14:09:04 +01:00
Stanko K.R. edb676cd2c Make remote IOs rewindable
This is required for active storage to work
2026-02-02 13:52:14 +01:00
Stanko K.R. 2dbbeffdcf Fix SSL issues when reading remote ZIP files
This is caused by our self-signed PureStorage certs. Once that's taken care of this commit can be reverted.
2026-02-02 13:16:59 +01:00
Stanko K.R. 5bbd7f633d Process everything in chunks
The old implementation loaded files into memory to provide an IO interface. This has the obvious downside of loading any file included in the import, e.g. a 10GB video file, into memory. ZipKit has no native IO object for reading but it provides all the necessary methods to implement one.
2026-02-02 12:36:48 +01:00
Stanko K.R. 992f15066b Replace RubyZip with ZipKit
ZipKit can read and write files directly from S3 which makes both imports and exports way more efficient in terms of disk usage.
2026-02-02 12:36:48 +01:00