How to make a machine listen to sound like a human?
A great advance in artificial intelligence technology has been achieved by modeling human systems.
Although artificial neural networks are mathematical models that can only roughly simulate how human neurons actually work, their application to solving complex and ambiguous real-world problems is far-reaching. Furthermore, modeling the structural depth of the human brain in a neural network opens up a wide range of possibilities for learning more meaningful meaning behind the data.
In image recognition and processing, inspiration from the complex and spatially invariant neurons in the convolutional neural networks (CNNs) of the visual system has also resulted in substantial improvements in our technique. If you’re interested in applying image recognition techniques to audio spectrograms, check out my article “What’s wrong with convolutional neural networks (CNN) and spectrograms for audio processing?”
As long as human perception surpasses that of machines, we can learn to benefit from understanding the principles of human systems. Humans are highly adept at perceptual tasks, and in the field of machine hearing, the contrast between human understanding and current AI technologies is particularly stark. Considering the benefits of taking inspiration from human systems in the field of vision processing, I suggest that we can apply neural networks to similar processes in the field of vision, and there will be benefits in the field of machine hearing.
How to make a machine listen like a human
The process framework of this article
In this series of articles, I will detail a framework for real-time audio signal processing using AI developed in collaboration between Aarhus University and smart speaker manufacturer Dynaudio A/S. It draws heavily from cognitive science, which attempts to combine perspectives from biology, neuroscience, psychology, and philosophy to better understand our cognitive abilities.
Cognitive properties of sound.
Perhaps the most abstract way to think about sound is how we humans understand it. While solutions to signal processing problems must work within the confines of low-level property parameters such as intensity, spectrum, and time, the end goal is often recognizable: to transform the signal in a certain way. that is cognitively meaningful to us about the meaning contained in The Sound.
For example, if one wishes to programmatically change the gender of the speaker of a discourse, the problem must be described in more meaningful terms before defining its lower-level characteristics. A speaker’s gender can be thought of as a cognitive attribute made up of many factors: the tone and timbre of speech, differences in pronunciation, differences in word and language choices, and understanding of how these attributes relate to each other. relate to gender.
These parameters can be described by lower-level features, such as intensity, spectral, and temporal properties, but only in more complex combinations can they form higher-level representations of meaning. This forms a hierarchy of audio features from which the “meaning” of the sound can be inferred. The cognitive properties of human voices can be thought of as being represented by the combined time series patterns of intensity, spectrum, and statistical properties of sound.
