LLM Clinical Failure Gallery
A gallery of ways language models fail on clinical questions, Arabic dialects foremost. Each case explains why the answer is wrong, the harm it could cause, and what a correct answer must contain.
- Year
- 2026
- Kind
- Healthcare · Python
- Status
- Shipped
Each case is one file: the prompt, an English gloss for any Arabic, the answer under critique, why it is wrong with every clinical claim cited, the potential harm, and what a correct answer must and must not contain. A 12-mode taxonomy separates general failures, such as under-triage, from four Arabic-specific ones: dialect misparse, code-switch confusion, translation drift and locale mismatch.
One case is a matched pair. The same warfarin question asked in English gets the right answer: keep vitamin K intake consistent. Asked in an Arabic dialect, it becomes "stop eating greens", and the interaction warnings disappear. A small Python build checks every case against the schema and refuses to build the static site if one breaks it.
All nine cases are illustrative reconstructions of documented failure patterns, labelled as such, not captured transcripts. Patients are synthetic. Every case is author-reviewed and cited, with no physician sign-off. It is a gallery, not a measurement, and says nothing about how often models fail.
