Meta announces the first automatic translation system of a language without written form

Artificial intelligence researchers at Meta are unveiling a breakthrough in machine translation this October 19, 2022. This is support for Hokkien, a language associated with the Hoklo ethnic group, originally from China, and spoken in certain regions of the Middle Kingdom as well as in Taiwan (of which it is one of the official languages) and in most countries of Southeast Asia. The system allows translation from Hokkien to English and vice versa.

The innovation is that it is an oral language, without a formalized written form. Machine translation systems usually rely on the written form only. Even in the case of voice translation on the fly, the system is divided into software bricks that manage the transcription of speech into text, the translation, and the vocal synthesis of the translated text into speech.

Another approach to translation

In contrast, Meta’s “Universal Speech Translator” project, announced in February, focuses on direct voice-to-voice translation. Its stated objective is to enable real-time translation of several hundred languages ​​into each other, which notably implies no longer systematically going through English as is the case for many systems today. The latter do not translate French directly into Japanese, for example, but from French into English, then from English into Japanese.

The idea is to be able to support certain less used languages, and especially for which there is no common text base with English that can be used for training AI models. The Meta teams rightly point out that of the 7,000 languages ​​currently in use around the world, more than 40% have no written form.

A demonstrator still far from real time

The first challenge was therefore to train this model. Researchers relied on Mandarin as an intermediate language to create baselines from Hokkien to English. They also used a voice encoding technique to create equivalences between Hokkien and English voice samples. It was also necessary to convert the said samples in order to be able to process them, then to assess the quality of the resulting translations. To do this, Meta used a scale called ASR-BLEU, which compares a machine translation transcript to a human translation.

Again, the lack of a written form of the language made the classical approach impossible, and so Meta developed a system to convert the Hokkien samples into a phonetic notation system called Tâi-Iô. They then calculated the “BLUE” score based on the syllables. The model, the data used to train it and the benchmark for judging the quality of the translation will be made available to the scientific community in open source.

The model is only able to translate a single sentence at a time in its current state, and only works to and from English, but Meta presents it as proof that this approach is viable. The researchers plan to extend this technique to other oral languages ​​and are hopeful that true real-time translation will eventually be possible.

In particular, they will make available a large base of speech-to-speech translations which have been “data mined” by a technique developed in-house, called Laser. It contains 418,000 hours covering 272 language pairs, including more than 8,000 hours of speech in Hokkien. The goal is for other researchers to create their own translation systems.

Selected for you

Stability AI, creator of the Stable Diffusion tool, raises $101 million

We would love to say thanks to the writer of this article for this remarkable web content

Meta announces the first automatic translation system of a language without written form

Take a look at our social media accounts along with other related pages