Skip to content

fix(archives): wrap the bare EOFError a truncated tar.gz raises - #3938

Open
Noor-ul-ain001 wants to merge 2 commits into
github:mainfrom
Noor-ul-ain001:fix/tar-truncation-eoferror
Open

fix(archives): wrap the bare EOFError a truncated tar.gz raises#3938
Noor-ul-ain001 wants to merge 2 commits into
github:mainfrom
Noor-ul-ain001:fix/tar-truncation-eoferror

Conversation

@Noor-ul-ain001

Copy link
Copy Markdown
Contributor

Problem

tarfile wraps most decompression failures in TarError, but a gzip stream that ends before its end-of-stream marker escapes as a bare EOFError from the gzip layer. EOFError derives from neither TarError nor OSError, so it bypassed all three tar handlers added with tar archive support (#3874):

Site Handler before
format probe in detect_archive_format except tarfile.TarError
tarfile.open in safe_extract_tar except (tarfile.TarError, OSError)
member iteration in safe_extract_tar except (tarfile.TarError, OSError)

A truncated .tar.gz — an interrupted download, a partially written file — raised a raw EOFError straight through the caller's error_type, so callers catching ValueError / ExtensionError / PresetError never saw it.

In specify workflow add the effect is worse than a traceback. Typer treats a bare EOFError as a Ctrl-D abort, so the whole diagnostic vanishes:

$ specify workflow add ./pkg.tar.gz      # truncated

Aborted.

The ZIP twin, given the same treatment, reports properly:

$ specify workflow add ./pkg.zip         # truncated
Error: Invalid workflow archive: Invalid ZIP archive: /tmp/pkg.zip

Fix

Route all three sites through a shared _TAR_DECOMPRESSION_ERRORS tuple so they stay in sync:

_TAR_DECOMPRESSION_ERRORS = (tarfile.TarError, EOFError, zlib.error)

zlib.error is included alongside EOFError: it is likewise neither a TarError nor an OSError, and can surface from a corrupt deflate block.

OSError is deliberately kept only on the two safe_extract_tar sites, which use it to report genuine I/O failures. Adding it to the probe would silently swallow those into "format mismatch" instead of the existing clean Invalid archive error, so the probe catches the decompression tuple alone.

After the fix, the tar path matches its ZIP twin:

$ specify workflow add ./pkg.tar.gz      # truncated
Error: Invalid workflow archive: Archive format mismatch: expected tar.gz, got
invalid/unsupported data

and domain error types wrap correctly again:

ExtensionError -> Invalid tar.gz archive: /tmp/...
PresetError    -> Invalid tar.gz archive: /tmp/...

Tests

Six regression tests in tests/test_download_security.py. tarfile decompresses lazily, so the leak surfaced at different sites depending on how much of the stream survived — the tests pin explicit byte counts to cover both:

  • 64 bytes — fails inside tarfile.open itself (covers the probe and the open site)
  • 512 / 2048 bytes — opens fine, fails during member iteration

Plus one test asserting the caller's error_type is honored, and one covering the safe_extract_archive entry point.

Verified test-the-test: all 6 fail against unmodified _download_security.py (DID NOT RAISE / raw EOFError), all pass with the fix.

tests/test_download_security.py    188 passed
tests/test_extensions.py           431 passed, 8 skipped
tests/test_workflows.py + test_presets.py   1403 passed, 27 failed

The 27 failures are the pre-existing Windows symlink-elevation class (OSError: [WinError 1314] A required privilege is not held by the client) — identical count and identity before and after this change on the same machine.

🤖 Generated with Claude Code

`tarfile` wraps most decompression failures in `TarError`, but a gzip
stream that ends before its end-of-stream marker escapes as a bare
`EOFError` from the gzip layer. `EOFError` derives from neither
`TarError` nor `OSError`, so it bypassed all three of the tar handlers
added with tar archive support (github#3874):

- the format probe in `detect_archive_format`, which caught only
  `tarfile.TarError`;
- `tarfile.open` in `safe_extract_tar`;
- member iteration in `safe_extract_tar`.

A truncated `.tar.gz` — an interrupted download, a partially written
file — therefore raised a raw `EOFError` straight through the caller's
`error_type`, so callers catching `ValueError`/`ExtensionError`/
`PresetError` never saw it. In `specify workflow add` the effect is worse
than a traceback: Typer treats a bare `EOFError` as a Ctrl-D abort, so
the command printed only "Aborted." with no diagnostic at all. The ZIP
twin reports "Invalid workflow archive: Invalid ZIP archive: <path>".

Route all three sites through a shared `_TAR_DECOMPRESSION_ERRORS`
tuple so they stay in sync. `zlib.error` is included alongside
`EOFError`: it is likewise neither a `TarError` nor an `OSError` and can
surface from a corrupt deflate block. `OSError` is kept only on the two
`safe_extract_tar` sites, which report genuine I/O failures; adding it
to the probe would silently swallow them instead.

Truncated tar.gz now reports the same clean, domain-typed error as the
ZIP path. Tests cover both the short prefix that fails in
`tarfile.open` and the longer ones that fail during member iteration —
`tarfile` decompresses lazily, so the leak surfaced at different sites
depending on how much of the stream survived.

Assisted-by: Claude Opus 5 (1M context)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Wraps tar/gzip decompression failures in caller-defined domain errors.

Changes:

  • Adds shared handling for TarError, EOFError, and zlib.error.
  • Adds truncated tar.gz regression coverage.
Show a summary per file
File Description
src/specify_cli/_download_security.py Handles decompression failures across tar probing and extraction.
tests/test_download_security.py Tests truncated tar.gz behavior and error wrapping.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Balanced

#: mid-member escapes as a bare ``EOFError`` from the gzip layer, and a corrupt
#: deflate block can surface as ``zlib.error``. Neither derives from
#: ``TarError`` or ``OSError``, so both bypass a ``(TarError, OSError)`` handler.
_TAR_DECOMPRESSION_ERRORS = (tarfile.TarError, EOFError, zlib.error)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — the zlib.error arm genuinely had no coverage. Fixed in ef49acc, with one correction to the claim the PR made about it.

zlib.error is reachable, but only at the two safe_extract_tar sites. tarfile converts it to ReadError when it surfaces while reading a member header, but the forward seek it performs to skip member data (tarfile.next, tarfile.py:2829-2834) sits outside that conversion, so a corrupt region past the first header escapes raw:

File "tarfile.py", line 2832, in next
    self.fileobj.seek(self.offset - 1)
File "gzip.py", line 580, in read
    uncompress = self._decompressor.decompress(b"", size)
zlib.error: Error -3 while decompressing data: invalid distance code

Reaching that seek needs members larger than the gzip read buffer — with small members the whole stream is decompressed during the first header read and the error gets wrapped, which is why single-byte corruption of the original 2.5 KiB fixture only ever produced ReadError. The new fixture uses two 256 KiB members at compresslevel=1 (a ~7 KiB archive, ~20 ms) corrupted past the midpoint so the first header still reads clean.

Four tests added: both safe_extract_tar sites (plain and with a caller-supplied error_type), safe_extract_archive with a caller-supplied error_type, and a guard asserting the fixture still reaches the module as a bare zlib.error — so if a future Python wraps it, that fails loudly rather than the coverage silently decaying into a duplicate of the EOFError cases.

Test-the-test: the three wrapping tests fail against unmodified _download_security.py with the raw zlib.error above, and pass with the fix.

Correction to the original PR description: it implied the probe in detect_archive_format needed the zlib.error arm too. It doesn't. tarfile.open alone only ever performs the header read that tarfile already converts, so I fuzzed 2800 corrupt archives against it and got zero bare zlib.error — only ReadError (610) or clean opens (2190). That arm is defensive at the probe site, not load-bearing. The tuple comment and the detection test now state this explicitly instead of implying coverage that can't exist. The probe still needs its EOFError arm, which the existing truncation test covers.

tests/test_download_security.py: 193 passed. The EOFError behavior and the deliberate OSError-only-on-extraction split are unchanged.

@mnriem

mnriem commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

Review feedback: the `zlib.error` arm of `_TAR_DECOMPRESSION_ERRORS` was
not exercised. Every regression added with the fix truncates a valid
deflate stream, which raises `EOFError`, so `zlib.error` could regress
independently of the EOF handling.

It is genuinely reachable, but only under a narrower condition than the
truncation cases. `tarfile` converts `zlib.error` to `ReadError` while
reading a member *header*, but the forward seek it performs to skip
member *data* (`tarfile.next`) sits outside that conversion, so a corrupt
region past the first header escapes raw. Reaching that seek needs
members larger than the gzip read buffer: with small members the whole
stream is decompressed during the first header read and the error is
wrapped. The new fixture therefore uses two 256 KiB members at
`compresslevel=1` — a ~7 KiB archive — corrupted past the midpoint so
the first header still reads clean.

Adds four tests: the two `safe_extract_tar` sites (plain and with a
caller-supplied `error_type`), the `safe_extract_archive` entry point
with a caller-supplied `error_type`, and a guard asserting the fixture
still reaches the module as a bare `zlib.error` — so if a future Python
wraps it, that fails loudly instead of the coverage silently decaying
into a duplicate of the `EOFError` cases.

Verified test-the-test: the three wrapping tests fail against the
unmodified `_download_security.py` with a raw
`zlib.error: Error -3 while decompressing data: invalid distance code`,
and pass with the fix.

Also corrects the scope claimed for the probe site. Fuzzing 2800 corrupt
archives never produced a bare `zlib.error` from `tarfile.open` alone,
because the only read it performs is the header read that `tarfile`
already converts. The probe's `zlib.error` arm is defensive, not
load-bearing; the tuple comment and a detection test now say so rather
than implying coverage that cannot exist.

Assisted-by: Claude Opus 5 (1M context)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Noor-ul-ain001

Copy link
Copy Markdown
Contributor Author

@mnriem Copilot feedback addressed in ef49acc.

It flagged that the zlib.error arm of the new error tuple had no test coverage — correct, and it turned out to be reachable at the two safe_extract_tar sites (tarfile wraps zlib.error during member-header reads, but not during the forward seek that skips member data). Added four regression tests, verified test-the-test.

It also surfaced that the PR description overstated the scope: the detect_archive_format probe cannot see a bare zlib.error (fuzzed 2800 corrupt archives — zero). That arm is defensive there, and the comment and tests now say so rather than implying coverage that can't exist.

Details in the review thread. tests/test_download_security.py: 193 passed.

@mnriem
mnriem requested a balanced review from Copilot August 4, 2026 12:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@mnriem mnriem left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix test & lint errors

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants