Artificial intelligence produced realistic Street Views with the help of street sounds

Artificial intelligence produces realistic street views, even though its material is just sounds recorded from the same street.

Soundscape and localization researchers at the University of Texas at Austin trained an artificial intelligence to create a street scene.

Professor Yuhao Kang trained a learning artificial intelligence with video clips assembled from still images and environmental sounds.

The audio-visual clips were about ten seconds long. They were taken from YouTube videos and generally depicted urban and rural scenes in North America, Asia and Europe.

Syväoppiva the algorithm learned which sounds corresponded to which objects in the images and the visual environment.

After training, the AI ​​had to produce images by itself. It received a hundred sound samples of certain environments depicting Street Views. The AI ​​produced one image per video.

People were shown these AI-generated images alongside two other generated images. At the same time, the subjects listened to the soundtrack on which the picture was based.

”

The street video shot at night was also made by artificial intelligence at night.

When respondents were asked to identify which of three pictures matched the soundtrack, they got it right eight times out of ten.

“Acoustic environments contain enough visual cues to create an accurate, recognizable streetscape,” Kang said based on the experiment in the University of Texas Bulletin.

“We can transform acoustic environments into lively visual presentations and sounds into attractions. When you close your eyes and listen, the sounds around you paint pictures in your mind,” says Kang.

“For example, the hum of traffic brings to mind a lively urban landscape. The rustling of leaves takes you to a peaceful forest.”

Kang is an assistant professor of geography and environmental science at the University of Texas and co-author of the study. The study was published by a scientific journal Computers, Environment and Urban Systems.

Artificial intelligence the produced images were examined in more detail on a computer.

In the pictures, the relationship between the sky, greenery and buildings strongly corresponded to the original authentic video clips. Even the architectural style of the buildings was preserved in the pictures.

Many times, the images produced by artificial intelligence also really reflected the light that prevailed when the picture was taken. The weather in the pictures was, for example, sunny or cloudy, and the street video shot at night was also made by artificial intelligence at night.

The night-time artificial intelligence recognized, for example, that traffic noise decreases at night. The sounds of insects also change when evening and darkness come.

Technique can provide applications. For example, a criminal investigator can use the application to get an idea of ​​where the audio recording related to the crime may have been made.

Artificial intelligence can be used to guide city planning and improve the quality of streets, researchers at the University of Texas state.

Before this artificial intelligence, street view and sound had been combined in a different way, as if in reverse order.

Artificial intelligence previously managed to create sound effects that corresponded to the view of a street or views filmed and recorded from other places, says the website New Atlas.