Crosspoint and Amplify, in charge of the metadata of the RTVE Archive

RTVE has been awarded to Crosspoint the implementation of AI (Artificial Intelligence) capabilities for the metadata of your Archive, a project in which the experience of the digital side of the company will be available, Cross media, and with the technology of Amplify.

The team of RTVE bet since 2020 on the automatic metadata in your Documentary Fund, due to the increase in the production of programs based on content from its archive, such as Iron and Chrome Bits, Trip to the TV Center o Vintage connection, as well as the implementation of video on demand platforms such as RTVE Play. After the completion of the previous service file, public radio and television launched a new tender for the management of the archive over the next few years. This service is of vital importance, given that the Corporation has set the objective of cataloging, in the medium term, 198,220 hours of material coming from the territorial centers.

Crosspoint (part of the group EN Medium), in collaboration with Amplify, has been awarded the tender, which will allow it to manage and provide the human and technological resources of the automatic metadata of audiovisual files from the RTVE Archive. The service that you will deploy with Amplify will perform automatic analysis of 20,000 hours: 18,000 audio in Spanish, 5,000 of the image contents y 5,000 multilingual audio. To address this project, Amplify will be deployed Seiri, an AI-based solution that automates and improves the management of audiovisual files, allowing content to be located and reused in seconds.

Multilingual audio, image and audio analysis

The technological service, whose deployment will be completed in September and will start October 1, covers a wide selection of processes in the field of audio treatment. Beyond audio-to-text transcription, Crosspoint and Amplify tools will take care of punctuation and capitalization of the transcribed text; Named Entity Recognition (NER); the extraction of keywords to identify the topics covered according to RTVE criteria; automatic classification of content, including its typology (interview, documentary, informative...); the creation of automatic summaries based on RTVE requirements; the identification of the speaker's gender, and the recognition of identity from the voice of public figures. In addition, they will carry out semantic segmentation, as an automatic minuteing of the contents; by news, for the analysis of news programs and magazines, and by speakers.

From the perspective of image analysis, the Crosspoint and Amplify solution is responsible for recognizing the presence of people in the scene; the recognition of the identity of public figures; the extraction of alphanumeric characters (OCR) in images; the identification of logos and brands; the detection of relevant objects; identification of scene types, and image description. Especially relevant is the analysis of news programs and magazines, given that the RTVE Archive has requested the identification and detection with their start and end time codes, sections and visual elements such as the header, headlines, sports block, weather block, credit titles, live signal, correspondent intros and the copy-text at the end of the broadcast.

In view of the analysis of multilingual audio, the system is responsible for the transcription of the languages of the Balearic Islands, the Basque Country, the Valencian Community, Galicia and Catalonia, and their translation into Spanish, whether they are pieces with the exclusive use of these languages or are bilingual. In addition, the system will also be used as a proof of concept (PoC) for specific audio and video analyzes such as the detection of effects such as applause, cheers or screams; the detection of violent content and hate speech; image search, or the detection and recognition of musical works.

Crosspoint will take care of the training of RTVE staff, who will be able to independently manage this tool with the support of a team available for consultations 12 hours a day and with a maximum resolution period of 48 hours.

Strategic alliance between Crosspoint / Crossmedia and Amplify

The collaboration for the RTVE Archive project between Crosspoint and Amplify is based on a collaboration framework agreement that will allow extending the metadata capabilities and automatic generation of summaries to all types of projects in the ecosistemas broadcast y OTT in all the markets in which both companies operate. In this way, as in the Spanish public radio and television project, solutions will be deployed such as Seiri, a “digital documentarian” capable of automatically analyzing audiovisual content and extracting a large amount of structured information through an accurate transcription of the audiovisual content, or GeNews, a solution based on generative AI capable of, through a form, generating an audiovisual piece that includes a script, image selection, basic editing and synthesized voice.

Victor Garcia, managing director de Crosspoint, values the alliance with Amplify very positively: "We are delighted to have reached a framework collaboration agreement with AMPLIFY, a company made up of top-level professionals, whom we have known for decades. They have put all their experience into the development of very advanced tools, which allow us to tackle highly complex projects such as the automatic metadata of the RTVE Archive. We hope to maintain this long-term relationship and target all types of markets, thanks to our global presence as part of ES Media Group. In this context, Memnon —also belonging to the group—plays a strategic role in the digitization of large-scale archives and the implementation of solutions based on artificial intelligence.”

On the part of Amplify, Aaron Lopez, founder and CEO, adds: "Surrounding ourselves with top-level, professional and solvent partners is essential to be able to fulfill our most ambitious projects. Therefore, joining Crosspoint and Crossmedia to transform the metadata of the RTVE Archive and plan new integrations with all types of televisions, platforms and radios is, without a doubt, the best decision we could have made."