gh-117807: Handle invalid UTF-8 in mimetypes map files

[harjothkhara]

Reading a MIME map file that contains invalid UTF-8 bytes in a comment raised UnicodeDecodeError, because mimetypes opens these files with encoding='utf-8' and strict error handling. A stray non-UTF-8 byte in a comment line shouldn't make the whole file unreadable.

Fix: open MIME map files with errors='replace' in both MimeTypes.read() and read_mime_types(), so invalid bytes are tolerated and the valid type mappings still load.

Added a regression test that reads a file with a non-UTF-8 byte in a comment and checks the following mapping parses correctly. Includes a NEWS entry.

Disclosure: AI-assisted; I've reviewed the change.

• Issue: mimetypes raises UnicodeDecodeError when map files are not unicode encoded #117807

[python-cla-bot]

All commit authors signed the Contributor License Agreement.

[bedevere-app]

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

[harjothkhara]

@sobolevn I saw that you’ve worked on mimetypes recently, could you please take a look at this fix?

I used errors='replace' so one bad byte in a comment doesn’t break the whole map file. I can switch to surrogateescape or skip the bad line if you prefer.

[harjothkhara]

@serhiy-storchaka on gh-117807 you suggested adding errors='replace' (or surrogateescape). This PR implements errors='replace' across the three read paths, with a regression test and NEWS. Can you please review when you have some time?

[kulikjak]

Hi, I don't have the explicit approval rights, but I still wanted to mention that I tested your change and it works as expected - the original issue we had is gone. Thanks!

[serhiy-storchaka]

I think it is safer to use 'surrogateescape'. For comments, there is no difference, but if non-UTF-8 sequence occurs in MIME type or extension, 'replace' will be lossy.

Also, the scope is wider than the title/NEWS suggest. The fix tolerates invalid bytes everywhere, not just in comments. This is fine, but please update the title/NEWS -- they should say "in MIME map files".

[harjothkhara]

Thanks, done. Switched both reads to surrogateescape, added a test for the type/extension case, and updated the title and NEWS to say map files instead of comments.

[serhiy-storchaka]

LGTM. 👍

[miss-islington-app]

Thanks @harjothkhara for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

[bedevere-app]

GH-155183 is a backport of this pull request to the 3.15 branch.

[bedevere-app]

GH-155184 is a backport of this pull request to the 3.14 branch.

[bedevere-app]

GH-155185 is a backport of this pull request to the 3.13 branch.