They say “seeing is believing”, but nobody ever proposes that “hearing is believing”. Yet how our brains make sense of what we see, and how our brains make sense of what we hear, seem to be accomplished more or less the same way. In each case, the brain starts by trying to figure out what it might be seeing/hearing, and tries to correlate what it actually sees/hears with its pre-determined idea. If the correlation is good, then our brains are able to conclude convincingly what it is that we are seeing/hearing. So, in order to see or hear something accurately, we don’t so much have to actually see it, or actually hear it. Rather what we need is an ensemble of evidence that allows our brains to make the necessary correlation in order for us to feel confident that we know what we are looking at or listening to.
There are many examples of this in action. The most obvious ones are ambiguous images, such as the one that is either a black table lamp against a white background or the white silhouettes of two faces looking at each other against a black background. I’m sure you can think of many other examples. When we look at such a picture, we cannot see both interpretations simultaneously. When we consider it to be a table lamp, we don’t perceive the faces. And when we see the faces, we don’t perceive the table lamp. For us to switch the way we see the image, we must consciously switch from one mode to the other. The more complex the image, the more effort is needed to switch our perception from one perspective to the other.
Over the last couple of weeks this has been illustrated to me in an interesting way. During this time, the hour I have been waking up in the morning has coincided with the time the sun rises, and starts to illuminate by bedroom. I have quite heavy curtains that do a pretty good job of keeping the sun out. But nonetheless the sun works way past the cracks. In doing so, it starts to illuminate the ceiling above my bed, throwing shadows of my curtain rail as it does so, making a pattern of dark lines across the ceiling. My ceiling is a flat white, and has an all-white 5-blade ceiling fan in the middle. At this time of year the fan is turned off. The angle of the fan blades is such that some of the blade faces are in the shadow of the encroaching sunlight, whereas one in particular faces it directly.
OK, you get the idea. Now, as the sun gradually rises, and illumination level rises, the fan blade which faces the sun just happens to take on the exact same shade of white as the ceiling behind it, and I cannot tell the two surfaces apart. I know exactly where that blade should be, because I can see the rest of the fan outlined quite clearly by its shadows. But over the course of about ten minutes, as the light gradually improves, what I see on my ceiling is a five-bladed fan, with one blade clearly missing. It is just invisible, as though it had been removed. My eyes do not detect the difference in tone of the white fan blade and the white ceiling behind it, so my brain tries to interpret the image as best as it can, and the best correlation it can come up with is the one with the missing fan blade. I know it is there, but try as I might I cannot perceive any indication of the presence of the apparently missing blade.
As the sun continues to rise, and the room gets brighter, eventually the optical illusion is replaced with the reality of the five-bladed fan, but at the point of transition an interesting thing happens. At a certain moment, if I concentrate hard enough I can see the emergence of some contrast around the edges of the “missing” fan blade. My brain can lock onto that and suddenly can “see” all five blades. However, like the aforementioned optical illusions, with some effort I can switch between perception modes. The fan can have either four blades or five.
This is where it gets interesting, though. I mentioned at the start that the curtain rail casts a shadow across the ceiling comprising a number of thin dark lines. Bear in mind that this is still a low-light situation, so the thin shadow lines are only just visible, but visible nonetheless. One of those shadow lines happens to run right through the disappearing fan blade. So when my brain sees the five-bladed fan, it sees that the shadow runs across the ceiling behind the fan blade but not across the fan blade itself. In other words, my brain recognizes that the fan blade obscures the shadow, so I see a shadow line interrupted by the 6 inches or so of the fan blade. Am I making myself clear? Good.
So what happens when I switch my perception mode to that of the four-bladed fan? In that case, my brain’s model has constructed no blade to obscure the shadow line, and so it expects to see the shadow line pass uninterrupted across the portion of the ceiling no longer obscured by the apparently missing fan blade. In short, when I visualize the five-bladed version of the fan, I see the shadow line interrupted by the fan, but when I visualize the four-bladed fan, I see the continuous shadow line. This is not a sub-conscious artifact - I can consciously switch my perception between the two versions of the apparent optical illusion. I am fully aware of the apparent contradiction which is that something which is quite obviously visible in one version becomes equally obviously invisible in the other version. Seeing is believing, indeed.
This holds some lessons for us in interpreting how we hear, if we are willing to accept this model of cognitive perception. It tells us that in order to be satisfied that we are hearing a certain thing, it is not sufficient to determine whether that thing is in and of itself audible. What we need is for that thing to make sense in the light of everything else that the brain is hearing at the same time. And I should say perceiving, rather than hearing.
As an example, humans have an uncanny ability to locate sounds in three dimensions. Not only can we locate something from left to right, but we can also locate it in terms of distance, and also in terms of height. How we are able to do this remains a topic of active research. The simple picture of how we hear is that we have two ears, and that our brains can infer the direction from which a sound is coming based on the time delay between the arrival of those sounds at our two ears. But this does not explain, for example, how we can perceive any vertical component to the localization.
This area of research has also shown up some remarkably interesting results. In the right conditions, test subjects can reliably differentiate between sounds originating from two points in space which are remarkably close together. When you crunch the numbers, the time difference between the arrival of the signals from those two locations is absolutely minuscule - of the order of 10 microseconds. This generates some tough problems, since in order to achieve that degree of temporal resolution, it is more or less a requirement that whatever is doing the detecting must have a bandwidth of around 250kHz. Considering that our hearing has a measurable upper limit of the order of 20kHz, this is problematic, and no theories yet exist to physically account for it.
However, some interesting observations do suggest that humans can perceive audio signals at frequencies considerably higher than 20kHz. By connecting a person up to a brain scanning device of some type, researchers have shown that the human brain can show a measurable response to audio signals at frequencies as high as 45kHz, even though the subject reports that they don’t hear a thing.
So if we base our models of audio reproduction theory solely upon simple stereo signals with a bandwidth of 20kHz, we may find ourselves unable to account for everything that practical experience throws at us. There are new things that we need to learn, but the fact that we don't know what they are is not an excuse for assuming they don't exist.
