fix(archives): wrap the bare EOFError a truncated tar.gz raises - #3938
fix(archives): wrap the bare EOFError a truncated tar.gz raises#3938Noor-ul-ain001 wants to merge 2 commits into
Conversation
`tarfile` wraps most decompression failures in `TarError`, but a gzip stream that ends before its end-of-stream marker escapes as a bare `EOFError` from the gzip layer. `EOFError` derives from neither `TarError` nor `OSError`, so it bypassed all three of the tar handlers added with tar archive support (github#3874): - the format probe in `detect_archive_format`, which caught only `tarfile.TarError`; - `tarfile.open` in `safe_extract_tar`; - member iteration in `safe_extract_tar`. A truncated `.tar.gz` — an interrupted download, a partially written file — therefore raised a raw `EOFError` straight through the caller's `error_type`, so callers catching `ValueError`/`ExtensionError`/ `PresetError` never saw it. In `specify workflow add` the effect is worse than a traceback: Typer treats a bare `EOFError` as a Ctrl-D abort, so the command printed only "Aborted." with no diagnostic at all. The ZIP twin reports "Invalid workflow archive: Invalid ZIP archive: <path>". Route all three sites through a shared `_TAR_DECOMPRESSION_ERRORS` tuple so they stay in sync. `zlib.error` is included alongside `EOFError`: it is likewise neither a `TarError` nor an `OSError` and can surface from a corrupt deflate block. `OSError` is kept only on the two `safe_extract_tar` sites, which report genuine I/O failures; adding it to the probe would silently swallow them instead. Truncated tar.gz now reports the same clean, domain-typed error as the ZIP path. Tests cover both the short prefix that fails in `tarfile.open` and the longer ones that fail during member iteration — `tarfile` decompresses lazily, so the leak surfaced at different sites depending on how much of the stream survived. Assisted-by: Claude Opus 5 (1M context) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Wraps tar/gzip decompression failures in caller-defined domain errors.
Changes:
- Adds shared handling for
TarError,EOFError, andzlib.error. - Adds truncated tar.gz regression coverage.
Show a summary per file
| File | Description |
|---|---|
src/specify_cli/_download_security.py |
Handles decompression failures across tar probing and extraction. |
tests/test_download_security.py |
Tests truncated tar.gz behavior and error wrapping. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Balanced
| #: mid-member escapes as a bare ``EOFError`` from the gzip layer, and a corrupt | ||
| #: deflate block can surface as ``zlib.error``. Neither derives from | ||
| #: ``TarError`` or ``OSError``, so both bypass a ``(TarError, OSError)`` handler. | ||
| _TAR_DECOMPRESSION_ERRORS = (tarfile.TarError, EOFError, zlib.error) |
There was a problem hiding this comment.
Good catch — the zlib.error arm genuinely had no coverage. Fixed in ef49acc, with one correction to the claim the PR made about it.
zlib.error is reachable, but only at the two safe_extract_tar sites. tarfile converts it to ReadError when it surfaces while reading a member header, but the forward seek it performs to skip member data (tarfile.next, tarfile.py:2829-2834) sits outside that conversion, so a corrupt region past the first header escapes raw:
File "tarfile.py", line 2832, in next
self.fileobj.seek(self.offset - 1)
File "gzip.py", line 580, in read
uncompress = self._decompressor.decompress(b"", size)
zlib.error: Error -3 while decompressing data: invalid distance code
Reaching that seek needs members larger than the gzip read buffer — with small members the whole stream is decompressed during the first header read and the error gets wrapped, which is why single-byte corruption of the original 2.5 KiB fixture only ever produced ReadError. The new fixture uses two 256 KiB members at compresslevel=1 (a ~7 KiB archive, ~20 ms) corrupted past the midpoint so the first header still reads clean.
Four tests added: both safe_extract_tar sites (plain and with a caller-supplied error_type), safe_extract_archive with a caller-supplied error_type, and a guard asserting the fixture still reaches the module as a bare zlib.error — so if a future Python wraps it, that fails loudly rather than the coverage silently decaying into a duplicate of the EOFError cases.
Test-the-test: the three wrapping tests fail against unmodified _download_security.py with the raw zlib.error above, and pass with the fix.
Correction to the original PR description: it implied the probe in detect_archive_format needed the zlib.error arm too. It doesn't. tarfile.open alone only ever performs the header read that tarfile already converts, so I fuzzed 2800 corrupt archives against it and got zero bare zlib.error — only ReadError (610) or clean opens (2190). That arm is defensive at the probe site, not load-bearing. The tuple comment and the detection test now state this explicitly instead of implying coverage that can't exist. The probe still needs its EOFError arm, which the existing truncation test covers.
tests/test_download_security.py: 193 passed. The EOFError behavior and the deliberate OSError-only-on-extraction split are unchanged.
|
Please address Copilot feedback |
Review feedback: the `zlib.error` arm of `_TAR_DECOMPRESSION_ERRORS` was not exercised. Every regression added with the fix truncates a valid deflate stream, which raises `EOFError`, so `zlib.error` could regress independently of the EOF handling. It is genuinely reachable, but only under a narrower condition than the truncation cases. `tarfile` converts `zlib.error` to `ReadError` while reading a member *header*, but the forward seek it performs to skip member *data* (`tarfile.next`) sits outside that conversion, so a corrupt region past the first header escapes raw. Reaching that seek needs members larger than the gzip read buffer: with small members the whole stream is decompressed during the first header read and the error is wrapped. The new fixture therefore uses two 256 KiB members at `compresslevel=1` — a ~7 KiB archive — corrupted past the midpoint so the first header still reads clean. Adds four tests: the two `safe_extract_tar` sites (plain and with a caller-supplied `error_type`), the `safe_extract_archive` entry point with a caller-supplied `error_type`, and a guard asserting the fixture still reaches the module as a bare `zlib.error` — so if a future Python wraps it, that fails loudly instead of the coverage silently decaying into a duplicate of the `EOFError` cases. Verified test-the-test: the three wrapping tests fail against the unmodified `_download_security.py` with a raw `zlib.error: Error -3 while decompressing data: invalid distance code`, and pass with the fix. Also corrects the scope claimed for the probe site. Fuzzing 2800 corrupt archives never produced a bare `zlib.error` from `tarfile.open` alone, because the only read it performs is the header read that `tarfile` already converts. The probe's `zlib.error` arm is defensive, not load-bearing; the tuple comment and a detection test now say so rather than implying coverage that cannot exist. Assisted-by: Claude Opus 5 (1M context) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
@mnriem Copilot feedback addressed in ef49acc. It flagged that the It also surfaced that the PR description overstated the scope: the Details in the review thread. |
mnriem
left a comment
There was a problem hiding this comment.
Please fix test & lint errors
Problem
tarfilewraps most decompression failures inTarError, but a gzip stream that ends before its end-of-stream marker escapes as a bareEOFErrorfrom the gzip layer.EOFErrorderives from neitherTarErrornorOSError, so it bypassed all three tar handlers added with tar archive support (#3874):detect_archive_formatexcept tarfile.TarErrortarfile.openinsafe_extract_tarexcept (tarfile.TarError, OSError)safe_extract_tarexcept (tarfile.TarError, OSError)A truncated
.tar.gz— an interrupted download, a partially written file — raised a rawEOFErrorstraight through the caller'serror_type, so callers catchingValueError/ExtensionError/PresetErrornever saw it.In
specify workflow addthe effect is worse than a traceback. Typer treats a bareEOFErroras a Ctrl-D abort, so the whole diagnostic vanishes:The ZIP twin, given the same treatment, reports properly:
Fix
Route all three sites through a shared
_TAR_DECOMPRESSION_ERRORStuple so they stay in sync:zlib.erroris included alongsideEOFError: it is likewise neither aTarErrornor anOSError, and can surface from a corrupt deflate block.OSErroris deliberately kept only on the twosafe_extract_tarsites, which use it to report genuine I/O failures. Adding it to the probe would silently swallow those into "format mismatch" instead of the existing cleanInvalid archiveerror, so the probe catches the decompression tuple alone.After the fix, the tar path matches its ZIP twin:
and domain error types wrap correctly again:
Tests
Six regression tests in
tests/test_download_security.py.tarfiledecompresses lazily, so the leak surfaced at different sites depending on how much of the stream survived — the tests pin explicit byte counts to cover both:tarfile.openitself (covers the probe and the open site)Plus one test asserting the caller's
error_typeis honored, and one covering thesafe_extract_archiveentry point.Verified test-the-test: all 6 fail against unmodified
_download_security.py(DID NOT RAISE/ rawEOFError), all pass with the fix.The 27 failures are the pre-existing Windows symlink-elevation class (
OSError: [WinError 1314] A required privilege is not held by the client) — identical count and identity before and after this change on the same machine.🤖 Generated with Claude Code