Skip to content

Fix 500 on whitespace-only hadith/chapter text (#158) - #3652

Open
wakqasahmed wants to merge 1 commit into
sunnah-com:masterfrom
wakqasahmed:fix/issue-158-whitespace-html-500
Open

Fix 500 on whitespace-only hadith/chapter text (#158)#3652
wakqasahmed wants to merge 1 commit into
sunnah-com:masterfrom
wakqasahmed:fix/issue-158-whitespace-html-500

Conversation

@wakqasahmed

Copy link
Copy Markdown

Summary

Fixes #158 (duplicate: #403). Bukhari chapters 89 & 96 (and likely others) returned:

{"error": {"details": "The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.", "code": 500}}

Root cause

fix_html() in text_transform.py strips the text and, if the result is empty, passes it straight to lxml.html.document_fromstring(). document_fromstring("") raises lxml.etree.ParserError: Document is empty. This is not a Flask HTTPException, so it is never caught by the app's @app.errorhandler(HTTPException) handler and bubbles up as an unhandled 500.

cleanup_text()/cleanup_chapter_title() only short-circuit when the raw field is falsy (None or ""), but a hadith/chapter text field can be a non-empty, whitespace-only string (e.g. a single space or newline). Such a value passes the if not text guard, survives the earlier re.sub calls unchanged, and only becomes empty after fix_html()'s internal .strip() — at which point lxml chokes.

This affects any hadith or chapter row where englishText/arabicText/englishBabName/arabicBabName/intro/ending is whitespace-only, which is the case for at least one row under Bukhari books 89 and 96 in production. This is a code robustness bug, not a data gap — the data is legitimately near-empty, the code should simply produce an empty string rather than crash.

Fix

Return "" early from fix_html() once the stripped text is empty, before calling into lxml.

Reproduction & verification

  • Reproduced locally via docker compose up (repo's sample MySQL dataset) by inserting a Bukhari book-89 hadith row with a whitespace-only englishText field, then hitting GET /v1/collections/bukhari/books/89/hadiths — got the exact same 500 / lxml.etree.ParserError: Document is empty traceback reported in the issue.
  • Applied the fix, re-hit the same endpoint — now returns 200 with "body": "" for the affected hadith, and unaffected rows are unchanged.
  • Removed the test row afterward; no schema/data changes shipped.

Tests

Added tests/test_text_transform.py covering whitespace-only input for all four cleanup functions, plus a sanity check that None/""/normal HTML are unaffected.

docker compose exec web python3 -m unittest tests.test_text_transform -v
# Ran 6 tests in 0.003s - OK

Also ran black --check and flake8 on the changed files — clean.

fix_html() passed whitespace-only text through lxml.html.document_fromstring
after stripping it to an empty string, which raises
lxml.etree.ParserError: Document is empty. That error is not an
HTTPException so it went unhandled, producing a generic 500.

Bukhari chapters 89 & 96 hit this because some hadith text fields are
non-empty but whitespace-only. Return empty string early instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Collection: Bukhari chapter 89 & 96 return an error code 500

1 participant