I’ve been staring at substitution reports from the leaderboard runs and something keeps nagging at me.
The word tingnan. It comes from the root tingin (to look), and the -an suffix gives you the object-focus form: tingnan. Regular morphology, same process that produces kainan, aralan, sulatan. But Filipinos also say tignan, freely and interchangeably, without giving it a second thought. The nasal cluster ngn gets flattened to gn, as if the root were tigin, which is not a word. Both forms coexist, and nobody particularly cares.
Prescriptively, tignan is nonstandard. Wiktionary gives it its own conjugation table but labels it as such. TagalogLang.com calls it a “widespread misspelling.” But a linguist would point out that this is just how language works: speakers tend toward more conservative forms, and the nasal cluster ngn is effortful. The simplification is natural. Whether you call it an error or an evolution depends on where you stand, and for everyday speech the question is entirely academic.
Here’s what the pipeline shows. The FLEURS reference has tingnan at four utterance sites. On two of those sites, eight out of ten ASR systems independently output tignan. The remaining two produced dignan and dingnan. Not a single system produced the reference form. Zero. The disagreement wasn’t between standard and nonstandard. It was between competing nonstandard renderings.
And the scoring pipeline has absolutely nothing to say about this. The GLM has 37 rules, all orthographic. The text normalizer is character-level. The normalizer I’ve been building handles orthographic variants. In practice, tingnan and tignan are just two ways of saying the same word, and you’d expect the pipeline to treat them that way. But the pair simply isn’t in the rules. Nobody put it there, because the rules were built from orthographic analysis, and this pair’s difference is phonological. It only surfaces as a distinct problem when you look at the derivation or, more to the point, when you listen closely to the acoustics.
So the pair just gets scored as an ordinary substitution error, same as if the system had output a completely unrelated word. That feels wrong.
It’s not just one word
I went looking through the substitution reports for more cases like tignan, places where the systems are reproducing a form that speakers actually say but that is morphologically wrong. The clearest candidate is meron for mayroon: ten events, seven out of ten systems agreeing on one site. Everyone says meron. It’s a contraction so entrenched that using mayroon in casual speech sounds stilted. The morphological analysis is clear (it’s a reduction of may + doon, and the full form is mayroon), but the reduced form is what people produce.
The substitution reports also turn up plural reduplication being dropped (maliliit → maliit), aspect marking differences (tatanggapin → tanggapin), prefix variation (ipinagmamalaki → pinagmamalaki). These are interesting but harder to classify. Is a system that outputs maliit for a speaker who said maliliit producing an adopted error, or is the speaker genuinely not reduplicating? I can’t tell from the alignment data alone. I’d need to listen.
What I can say is that the FLEURS reference is not a reliable anchor for sorting any of this out. Hinarahap, magpaliwanang, mamamayahag: these sit in the reference as morphological errors (metathesis, misplaced nasal), and all ten systems unanimously output the correct hinaharap, magpaliwanag, mamamahayag. Those are labeling problems, a different issue that I’ve written about separately. But they compound this one, because any equivalence mechanism has to account for the fact that the reference can be wrong in the same ways speakers are wrong.
What I’ve been reading
I went looking for whether anyone in the ASR literature has dealt with this, and the answer is: sort of, but not quite.
The closest thread is “transcription policy.” A paper from Nyra Labs (July 2026) argues that when neither the model output nor the reference makes its transcription style explicit, WER conflates content errors with style differences. Toyin et al. at MBZUAI showed that different model architectures actually specialize in different transcription targets (verbatim vs. intended), so rankings shift depending on which reference you score against.
But those papers are about disfluencies. Fillers, repetitions, false starts. A speaker who says tignan is not being disfluent. She means to say tignan. There’s no self-correction in the signal, no hesitation, no repair. The verbatim/intended distinction doesn’t apply here because the speaker’s intention and her production are perfectly aligned. They just happen to diverge from the prescriptive standard.
The sociolinguistic ASR bias literature is closer. Markl’s 2026 analysis of Newcastle English showed that ASR errors are socially patterned, driven by dialect-specific phonology. The Swiss German ASR work frames the output choice as dialectal writing (faithful to the acoustic signal, messy) vs. normalized writing (consistent, distant from what was said). The Ramsa corpus for Emirati Arabic chose to transcribe each instance as actually produced, no imposed standardization.
But tignan is not a dialect feature. It doesn’t mark a social group or a region. It’s a simplification that has spread across the entire speech community, standard and nonstandard speakers alike. There’s an Inquirer (Philippine newspaper) piece about exactly this feedback loop: media are the biggest users of language, nonstandard forms in broadcast get multiplied nationwide, and then everyone adopts them. ASR systems trained on that same broadcast and web-scraped text are just another amplifier in the loop.
I haven’t found anyone writing about this specific class of problem: forms that are analyzable against a prescriptive standard, used interchangeably with the standard form by the majority of speakers, and now baked into both reference transcripts and model training data. It falls between the cracks of the existing frameworks.
What I don’t know yet
The scoring pipeline currently handles this by not handling it, which is itself a policy. Every tingnan/tignan pair gets scored as a substitution. Systems that faithfully transcribe what speakers actually say get penalized. Systems that happen to produce the standard form get rewarded. And when the reference is the one carrying the error, the whole thing flips.
I could add tignan → tingnan to the GLM tomorrow. Mechanically trivial. But the design question underneath is not trivial at all.
There’s the diagnostic angle: if I equate them, I’m suppressing a potentially useful signal. Which systems have internalized Filipino morphology vs. which ones have just memorized the surface frequency of tignan in their training data? A system that outputs tingnan when the speaker said tignan might be demonstrating deeper linguistic competence. Or it might be hallucinating a form the speaker didn’t produce. Both readings are defensible.
But there’s a more fundamental problem, and it has to do with the primary responsibility of an ASR system: faithfully representing the acoustic signal. Consider a conversation where the topic is precisely the difference between tignan and tingnan. A linguistics class. A language podcast. A KWF panel discussion. If the ASR normalizes every tignan to tingnan, anyone reading only the transcript would be completely lost. The discussion would reference a distinction that the transcript has erased. The system would have prioritized prescriptive correctness over its most basic job: telling you what was said.
This is the argument that gives me the most pause. An ASR system is not an editor. Its first obligation is to the acoustic signal, and any normalization that collapses a distinction in the signal is overstepping, however rarely that distinction matters in practice.
What I think I actually need is a new field in the normalization schema: prescriptive status. A way to record that a variant is not an alternative spelling but a form that diverges from the prescriptive standard while being widely produced and understood. Whether the pipeline then equates, penalizes, or reports these pairs separately is a downstream decision. But right now the tooling can’t even represent the distinction, so the decision is being made by omission.
I’ll sit with this a bit longer.