Cantonese Elders’ Life Stories Could Reach Families Sooner With an AI First Draft

“Mum’s from Hong Kong.” In one interview, an elder explains the two ways his family said “to eat”: his mother one way, in Cantonese, his father another, in Taishanese. Then he reaches the word for bowl and says it several ways in a row. “We said, oh, yeah, okay. That is how it is, right?”

That moment comes from 12 hours and 42 minutes of interviews recorded for the Cantonese Heritage and Culture in Aotearoa New Zealand oral history project. Five community elders, all 65 or older and all born in New Zealand or arrived there before 1980, sat down with interviewers from their own community and agreed to share their recordings. In a preprint posted July 21, 2026, Sidney Wong of the University of Otago and members of the New Zealand Chinese Association report what happened when they handed those recordings to Whisper, an open-source speech recognition model.

The model produced a rough first transcript of the whole collection in five hours. The team estimates a person would need about 508 hours to transcribe it by hand. By raw computer run time, that is less than 1 percent of the manual effort, before anyone checks the result.

A language that has been slipping away

Cantonese speakers have lived in New Zealand since the 1840s, and the 2023 Census counted 54,417 of them, making Cantonese the seventh most spoken language in the country. Yet the paper describes heavy language loss. A survey of Cantonese-speaking families in Wellington in the 1990s found that across three generations, 70.96 percent of family members no longer spoke Cantonese. The authors point to history: the Chinese Poll Tax introduced in 1881, and a complete ban on language teachers entering the country.

Oral histories are one way the community is pushing back, and the paper notes that in community oral histories like these, the people telling the stories remain in control of them. The recordings remain the intellectual property of the speakers. They will be released through a public archive, and one elder chose to have theirs released only after their death.

Where the machine got lost

To check the drafts, the team transcribed one 9 minute and 33 second interview segment by hand: 1,198 words that move between English, te reo Māori, Cantonese and Taishanese. The best Whisper setting scored a word error rate of 12.10, roughly 12 errors for every 100 words, but the authors say that score came at the expense of the non-English words.

That number hides the hard part. The English came through well. The rest often did not. When the elder said the Taishanese word for bowl and then explained it in English, one model wrote the Taishanese word as “Wan” and the English word “bowl” as “bull.” None of the models could tell Cantonese from Taishanese, some invented words outright, and when left to identify the language on their own, six of the eight models decided the main language of the interview was te reo Māori. The top-scoring setting was one of them.

What a rough draft is worth

A survey of linguists cited in the paper found that one hour of recordings can take up to 40 hours to transcribe word by word. For a volunteer community project, that is the difference between an archive people can search and read and a box of audio files. The authors conclude Whisper is useful as a first pass, but only for the English parts, and that time and money should go to people who can correct the Cantonese and Taishanese.

That is a hopeful division of labor. The machine types the easy stretches. Fluent speakers spend their scarce hours on the words only they can hear, the ones that carry a family’s village, its dialect and its dinner table. The team’s next step is to try adapting the model to this kind of mixed-language speech, which they say will mean consulting the speakers and the wider community first.