The Topo-Speech sensory substitution system as a method of conveying spatial information to the blind and vision impaired frontiersin.org/articles/10.33

@seeingwithsound Topo-speech is an interesting concept to represent space. Look forward to the full article to learn more :ablobderpy:
One question I have with visual to auditory substitution is how it may interfere with normal environmental sound. There are a lot of cues from the environment that such vision-to-sound systems would be masking.

@hkl Blind users typically use bone conduction stereo headphones that rest on the temples and do not block the ears (which otherwise interferes also with echolocation), and modest audio volume further helps to minimize masking. Sound characteristics are quite different from other sounds such as speech, and that helps too.

@seeingwithsound Thanks! I didn't know that. Doesn't bone conduction has a different frequency range as with airborne sounds? I remotely remember that it loses high frequency range. If so, does that pose a limit to what can be conveyed through the system?

@hkl Human hearing sensitivity for frequency changes is ~constant only on a log frequency scale, such that the higher frequencies add little in terms of extra information capacity, plus for the elderly loudness sensitivity drops off rapidly >6kHz. The vOICe therefore uses [500-5000]Hz, and bone conduction is fine for that. Bigger disadvantage is poor stereo, but in practice some sound still leaks over-the-air and suffices for directional hearing.

@hkl The red thing hanging somewhat behind the back of the head of a Russian blind user of The vOICe vision glasses in this YouTube video are bone conduction stereo headphones from Aftershokz youtube.com/watch?v=AZ1xigijrx

@seeingwithsound Thanks! It seems the person has good success using the glasses.

@hkl Yes, this is way beyond what has thus far been demonstrated with retinal implants and cortical implants for vision (let alone Neuralink...).

@seeingwithsound
So what is the limiting step for this to move forward to more users? Is it difficulty in mastering its use or does it need more development to provide better sound information?

@hkl Cost of 1-on-1 training support, substantial training effort, attitude and motivation, lack of awareness of and belief in possibilities, no immediate visual percepts (flashes of light), fragile smart glasses, ... or the short answer: dunno. :-) You never know if or when a paradigm shift - away from electrodes - will take place.

@seeingwithsound Have you thought about using video game to train people. I was at a seminar where Daphne Maurer gave a talk describing her work that showed visual improvement in people with amblyopia. Her idea was that motivation is key for perceptual learning. Here's her publication if interested.
pubmed.ncbi.nlm.nih.gov/231936

@hkl Good ideas, but:

1. I suspect that 1-on-1 support was (via motivation) key to any successes in the cited paper: not scalable. Similarly, very few people master a foreign language without 1-on-1 interaction with a human teacher: not scalable.

2. Making a good game is a huge effort with uncertain outcomes: a resource hog and gamble.

The Russian successes came through many hours of 1-on-1 training support, which is hard to make (commercially) sustainable: trainers want to eat too.

Follow

@seeingwithsound Good points. I agree that nothing beats 1 to 1 support, if resources are available.

Sign in to participate in the conversation
CleverLibre Social

CleverLibre Social is an inclusive social instance for open discussion, learning, and community.
All cultures welcome.
Hate speech and harassment strictly forbidden.