Fix 500 on whitespace-only hadith/chapter text (#158) - #3652
Open
wakqasahmed wants to merge 1 commit into
Open
Conversation
fix_html() passed whitespace-only text through lxml.html.document_fromstring after stripping it to an empty string, which raises lxml.etree.ParserError: Document is empty. That error is not an HTTPException so it went unhandled, producing a generic 500. Bukhari chapters 89 & 96 hit this because some hadith text fields are non-empty but whitespace-only. Return empty string early instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #158 (duplicate: #403). Bukhari chapters 89 & 96 (and likely others) returned:
{"error": {"details": "The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.", "code": 500}}Root cause
fix_html()intext_transform.pystrips the text and, if the result is empty, passes it straight tolxml.html.document_fromstring().document_fromstring("")raiseslxml.etree.ParserError: Document is empty. This is not a FlaskHTTPException, so it is never caught by the app's@app.errorhandler(HTTPException)handler and bubbles up as an unhandled 500.cleanup_text()/cleanup_chapter_title()only short-circuit when the raw field is falsy (Noneor""), but a hadith/chapter text field can be a non-empty, whitespace-only string (e.g. a single space or newline). Such a value passes theif not textguard, survives the earlierre.subcalls unchanged, and only becomes empty afterfix_html()'s internal.strip()— at which point lxml chokes.This affects any hadith or chapter row where
englishText/arabicText/englishBabName/arabicBabName/intro/ending is whitespace-only, which is the case for at least one row under Bukhari books 89 and 96 in production. This is a code robustness bug, not a data gap — the data is legitimately near-empty, the code should simply produce an empty string rather than crash.Fix
Return
""early fromfix_html()once the stripped text is empty, before calling into lxml.Reproduction & verification
docker compose up(repo's sample MySQL dataset) by inserting a Bukhari book-89 hadith row with a whitespace-onlyenglishTextfield, then hittingGET /v1/collections/bukhari/books/89/hadiths— got the exact same 500 /lxml.etree.ParserError: Document is emptytraceback reported in the issue.200with"body": ""for the affected hadith, and unaffected rows are unchanged.Tests
Added
tests/test_text_transform.pycovering whitespace-only input for all four cleanup functions, plus a sanity check thatNone/""/normal HTML are unaffected.Also ran
black --checkandflake8on the changed files — clean.