TSA and the Basque Government advance in the automatic dubbing of content into Basque with AI

The Basque Government through the Public Information Society of the Basque Government (EJIE) y Telefónica Audiovisual Services (TSA) have undertaken a pioneering proof of concept that has aimed to analyze the current possibilities of automatic dubbing of content from Spanish to Basque.

To carry out this project, TSA has relied on Etiqmedia, technological partner specialized in artificial intelligence. Together, they have dubbed three contents of Movistar+: and docurreality, a own production series and some children's cartoons. Precisely, this last category has been the one that has yielded more satisfactory results, given that technology has been able to reproduce with great fidelity the voice of the characters, rhythm and expression of the dialogues.

TSA believes that this proof of concept opens the door to fast and automatic exploitation of this type of processes, counting, yes, with a final quality control before export. Likewise, this “milestone” opens the doors to “a new dimension and monetization” of various children's content that can be found in the archives of production companies and distributors. By being able to make the leap to Basque semi-automatically, they will be able to start making money, something that until now was impossible due to the “high cost of traditional manual dubbing.”

The serie and the docurreality have been “less satisfactory,” in the words of TSA. In the original of Movistar+, although the automatic dubbing has faithfully translated, dubbed and reproduced the cloned voice of the main character, it has not been able to capture “feeling, rhythm and expression” of the dialogues, especially in the shorter sentences. Even more complex has been the case of docurreality, with errors in the pronunciation and understanding of the context. TSA has counted on the collaboration of the project for the three tests come back, a speech synthesis service in Basque offered by the Basque Public Society in the form of two female voices and one male voice in Basque.

Automatic dubbing: a reality

For TSA, this proof of concept yields one main conclusion: automatic dubbing from Spanish to Basque is a reality, especially in the case of cartoon, but there is still a long way to go to improve its performance in certain content. The Telefónica company and Etiqmedia have announced their willingness to continue working in this line with the aim of bringing these processes to Basque and other languages.

In the words of Asier Anitua, manager of Telefónica Servicios Audiovisuales: "At Telefónica we are committed to the promotion and dissemination of Basque and the official languages ​​of the Spanish territory. This proof of concept is a sample of our commitment to this language. We are convinced that automatic dubbing can be a very useful tool to bring Basque closer to a greater number of people."

To achieve these ambitious objectives, it will be essential to continue studying in depth the variables that will allow a piece to end up being dubbed and translated automatically maintaining the identity and quality of the original work. Among these factors, the content type, the training data quality o to technology used, since the specific AI developments used considerably influence the results.

Antonio Leon, CEO of Etiqmedia, adds: “Our exclusive technology shows that we can already achieve high quality results for certain content, proof of which are the cartoons that can now be exploited for our clients in several languages.”