They create two translation platforms from Mexican sign language to Spanish

Using artificial intelligence (AI) techniques, two platforms were developed to promote the learning of Mexican Sign Language (LSM). These are Daktilos, a tool that automatically translates from this language to Spanish, as well as Sordbo, which helps cafeteria customers place orders using manual signs.

In an interview with this medium, Irvin Hussein López Nava, researcher at the Center for Scientific Research and Higher Education of Ensenada (Cicese), who has led both projects, explained: They are ambitious initiatives because if translation between oral languages ​​is complicated, it is even more so when it comes to the conversion from an oral language to a sign language, where the elements of communication are different..

Daktilos – whose name comes from Greek dates, finger science – was developed by Jesús Gortarez Pelayo, a student at the University of Baja California. It is an interactive website through which the country’s sign language alphabet is taught.

“If we take into account that only 30 percent of Spanish words can be represented by a sign,” said López Nava, “knowing how to illustrate each letter is very useful, since it allows us to communicate concepts that do not have a predefined sign.” An example is proper names; When someone introduces themselves for the first time, they have to spell it.

A relevant characteristic of Daktilos is that it has an interaction system through which it rates from one to 10 how well a person signs. This is possible because one of the main AI tools it uses is computer vision, which allows machines (cell phones, iPads, personal computers) to identify a part of the human body and follow its movements.

In order for these movements to be translated according to sign language, the scientists also applied an algorithm based on artificial intelligence with more than 20 thousand images for each of the 30 letters that make up that alphabet to detect what is being signed.

Daktilos allows the learning of frequent words in colloquial speech such as mom or dad; It contains the hangman game, a pastime in which participants try to guess a hidden word letter by letter.

This project, tested in schools and scientific dissemination activities, is about to be deployed on a server to be used by the public from four years of age.

This system benefits the speakers, it allows them to know how to communicate when they are in front of a deaf community. It’s about promoting inclusionsaid the academic.

Alliance with academia

Meanwhile, Sordbo – whose name has its origins in the play on words between deaf and sorbo – is a platform that allows cafeteria customers to place their order in Mexican sign language. Its development was led by Alan Montesinos Scott, a student at the Baja California University.

Like Daktilos, this initiative arose from an alliance between the Cicese Department of Computer Sciences and the Regional Association of the Deaf of Ensenada, which has a cafeteria in the Jimena Center, in Ensenada, where classes and workshops are given to children, young people and adults with different abilities. At that place, the barista is deaf, and the only way to interact with him is by signing.

To take advantage of technology and encourage listeners to use sign language, Sordbo was developed, which allows customers to select menu items from a cell phone or computer, and shows them the hand signs they need to make to address the barista.

Its purpose is to promote deaf culture, so that there is no segregation. We believe that this system can be the door for the public to become interested in learning sign language.reported López Nava.

After creating Daktilos and Sordbo, which are part of the educational and commercial field, López Nava’s team is focused on developing an initiative that addresses the medical and emergency context. For this reason, they work on a base of 100 words to generate phrases that help express physical discomfort and ask for help.

If these tools had been attempted 10 years ago it would have been impossible. Today they can be done because there are multiple advances in artificial intelligence tools and mobile devices with high-quality cameras, next-generation sensors and very high processing capacity.he finished.