What is Metamorphic Testing?
Metamorphic testing, told as a story about checking a translation with no single right answer by a relationship: repeat the sentence, and the translation should repeat too.
The proofreader has no known right answer to check the translation against.Translation often has no single correct answer.
The usual trick — comparing to an answer key — doesn't work here.So is there no way at all to check the translation?
Metamorphic testing checks the translation by the relationship between input and output, even with no known answer.Feed the sentence twice, and the translation should come out twice too.
Like 'repeat the sentence, and the translation repeats', relationships between input and output are set in advance and checked.If a relationship breaks, it signals a fault in the translator — even with no answer key.
A badly chosen relationship can miss a real bug.When the answer is known, comparing to it directly is still more precise.
Metamorphic testing finds bugs by checking a relationship — if the input changes a known way, the output must change a known way too — even with no answer key.
Metamorphic testing is a technique for programs with the 'oracle problem' — where there's no known correct output, or no single one to pin down. For systems like translators, search engines, or ML models, it defines 'metamorphic relations' that the output must satisfy when the input is transformed in a known way, and detects faults when those relations are violated.
When grown-ups say it
- Metamorphic Testing
- Checking by relationship, with no answer key. Used when the oracle problem exists.
- Oracle Problem
- A situation with no answer key, or no single answer to pin down. Common in translation, search, and ML.
- Metamorphic Relation
- The rule that a given input change implies a given output change. Must be defined ahead of time.
- Regression Test
- The proofing that checks the plot hasn't shifted. More common when the expected result is known.
- Hypothesis Testing
- Telling a real difference from coincidence, with numbers. A different way of handling uncertainty. → which cover really sells better
- Black-box Testing
- Checking only the result, without looking inside. Metamorphic testing belongs to this family. → looking inside, or just reading the outside
- Unit Test
- The usual proofing that compares against a known answer. More precise when the right answer is actually known.
- AI Hallucination
- Producing a plausible-sounding but wrong answer. The same metamorphic idea helps verify AI output with no answer key. → the AI field's plausible lies