Title of Invention	ARRAY BASED TACTILE VISION FOR THE BLIND PEOPLE USING MULTI-DIRECTIONAL ULTRASONIC SCANNING OF ENVIRONMENT
Abstract	A vibrotactile visual system for the visually impaired people, based on a novel method of echolocation also useful for the other intelligent devices where understanding of obstacle distribution in the surrounding environment is necessary is disclosed. One of the typical implementation of this system comprises an array of ultrasonic transceivers directed in different direction (5, 14), an array ofvibrotactile actuators (18) and electronic circuit module. As this is does not require any bulky laptop or computer for its functionality it is quite lightweight and low cost. This device having a vibrotactile visual interface to the blind person creates very less audio interference with his hearing capability. User also may use it as a supportive traveling aid while using their conventional traveling to get trained on this new system. After getting trained and confidence on this system he may completely rely on this system for his traveling and normal activities. As for using this system users need not to hold anything on his hand they can use their hand for their desired activities.

Title of Invention

ARRAY BASED TACTILE VISION FOR THE BLIND PEOPLE USING MULTI-DIRECTIONAL ULTRASONIC SCANNING OF ENVIRONMENT

Abstract

A vibrotactile visual system for the visually impaired people, based on a novel method of echolocation also useful for the other intelligent devices where understanding of obstacle distribution in the surrounding environment is necessary is disclosed. One of the typical implementation of this system comprises an array of ultrasonic transceivers directed in different direction (5, 14), an array ofvibrotactile actuators (18) and electronic circuit module. As this is does not require any bulky laptop or computer for its functionality it is quite lightweight and low cost. This device having a vibrotactile visual interface to the blind person creates very less audio interference with his hearing capability. User also may use it as a supportive traveling aid while using their conventional traveling to get trained on this new system. After getting trained and confidence on this system he may completely rely on this system for his traveling and normal activities. As for using this system users need not to hold anything on his hand they can use their hand for their desired activities.

Full Text	Array based tactile vision for the blind people using multi-directional ultrasonic scanning of environment This invention relates to improvement of the devices used as electronics traveling aids for blind people. Till date electronic traveling aids for the blind people is not so much popular among the blind people specially in the developing countries like ours, because of their higher cost and the heavier weight So most often they are traditionally relying on the simple or folding cane as a mobility aid. But such cane can't give them the entire morphology of the environment in front of them. There are plenty of traveling aids already designed for the blind people some of them are electronic specifically utilizing the LASER or the ultrasound, most of mem are stimulating the blind person raising some alarm, may be audio or tactile depending on the presence of the obstacle in front of him. Some of mobility aids are delivering continuous audio stimulation to the blind person, which also represents a gross environment in front of the blind person. But this system highly interferes with the input of the most vital sensory system for the blind people - hearing capability and at the same time it requires a prolong training to understand the pattern of the audio stimulation to correlate the obstacle distribution in the environment. Some mobile computer (or simple laptop) based systems are readily available with the capability of speech-alarm but their cost and weight is very high. Another major problem in the detection of the obstacle and its distribution is to identify die rank of the obstacles i.e., either a direction of the environment is congested with obstacles or the direction has some partial obstacle. So we need to assign different weight on different type obstacle distribution rather than on an individual obstacle. In this project we have developed an ultrasonic tactile visual system with the capability of interpretation of obstacle distribution in the environment to act as a mobility aid for blind. We had set our goal to make this system light-weight and low-cost (raw material cost at the development stage is around Rs.5000), so that it can be used widely for the rehabilitation purpose for the blind people. So we have used the microcontroller based DSP, instead of using a bulky laptop in the design of the digital electronic circuitry and embedded software. To interface this system with a blind person we are incorporating array based vibro-tactile vision. It enables him to enjoy lesser audio interference to their normal hearing capability. We have designed our system such that the blind person needs not to hold the device in his hand, so that he can utilize his hand for other activities. Most of all we have made the instrument and its operating procedure such that any blind person can utilize it without any long term training at least for the localization of die distinct obstacle, whereas a proper training enables him to understand the morphology and distribution of the obstacles surrounding him. In addition to the above die data acquisition part used in the above mentioned system is also quiet useful for position localization of the mobile robots, where we need to implement a little bit different type of neural-network in the DSP chip. This feature will also enable our system to raise some audio-alarm to the blind person. Brief description of the figures: Fig-l - Fig-5 used for the explanation of the working principle of our system. Fig-6 shows the schematic of data acquisition block. Fig-7 shows me schematic of synchronization and processing block. Fig-8 shows the schematic of vibro-tactile actuator driver block. Fig-9 shows the PCB layouts of the data acquisition and processing block used in this system. Fig-10 & Fig-l 1 shows the model of vibro-tactile array and its driver circuit layout designed by us. Detail Description of the above system: a) Transmission and reception block: We know that the continuous transmission of ultra sound creates standing wave by me superposition of me transmitted beam and the reflected beam, so continuous transmission is not suitable for assessment of the power of reflected beam. For this reason pulsed transmission of ultrasound is one of the easiest mean for assessing die power of the reflected beam and the time required to turn-back depending on die distance of the reflecting obstacle. The following scheme shows die scheme for die distance assessment based on ultrasound traversal time. The system we have designed also processes the attenuation of the received ultrasound pulse to assess the distance of the obstacle. It has 6 (expendable up to 8 or may be reduced as per the specific requirement) transmission channels directed in a fashion shown in Fig-1 which activates in a pattern of TDM. Here to enhance the power and efficiency we have designed sinusoidal burst or pulse generating circuit along with a matching step-up transformer. Fig-1: Directivity of the transducers mounted in the head band So we can see that the six transmitters are directed in six different directions for transmitting ultrasound pulses one by one. For each transmitted ultrasound pulse six detectors receives reflected ultrasound pulses of different intensities and at different times. These 6 X 6 = 36 intensity & time pairs bear impotent information regarding the locations of the obstacle in the environment of the ultrasonic visual system. Now to process the acquired information we have used a PIC series microcontroller - PIC16F877 or PIC18F452 along with a 16-bit DSP chip - dsPIC30F2011 or higher (optional for inbuilt decision making feature especially for mobile robots), which not only keeps the synchronization among the various functionalities of the visual system but also extracts the conclusion from the obtained data. The blind person is continuously informed about the acquired information by utilizing vibro-tactile vision. When ever the client sends a requests to this system to know the distribution of obstacles by specifying a directions of their own interest, the inbuilt decision will deliver the information it him (optional). b) Signal conditioning and svnchrinization block The technique we have used here to transmit sequential ultrasonic pulse or burst is basically consisting of three fundamental building blocks. First of all we have designed a 40 KHz sine-wave generator using the IC XR2206. Then it is sequentially demultiplexed in 6 different channels of ultrasonic transmitter block. Here each channel of the demultiplexer is controlled by the control block such that, only up to 16 cycles (programmable according to the need) of the 40 KHz signal passes to the ultrasonic transmitter block and then the transmission remains suspend for a duration of 60 mSec for that channel and the next channel is selected. This process is continued in a cyclic fashion. During this 60 mSec of null transmission period all the 6 (expendable up to 8) ultrasonic receivers receive the reflected ultrasonic pulses. Each of the receivers consists of three building locks: the first one is the high pass filter to remove 50 Hz noise followed by a half wave integrator. The second one is die nearest obstacle determination circuit and both of them are controlled by third control-block. During the transmission period of 0.2 to 0.4 mSec all the integrating capacitors are made discharged to keep it ready for the integration during next 60 mSec. At the end of the 60 mSec of integration all the output voltages of the receiving channels are internally digitized by the microcontroller. This system is also associated with a temperature sensor to correct the variation of sound- velocity in air, due to temperature variation. As in any direction there may lie several partial obstacles so each receiving channel may receive more than one reflected pulse for a single pulse transmitted from any one of the receivers. For this reason each channel is associated with a nearest obstacle monitoring block which interrupts the microcontroller with a pre-assigned priority, whenever the first reflected pulse is received in that channel. Now the microcontroller determines the time required to obtain the ultrasonic pulse reflected from the nearest obstacle using an internal 16-bit timer. So at the end of the 61 mSec (approx) we are accompanied with the (6 X 6 X 2 = 72) number of 16-bit data. Now if we are going to utilize only the neural-network (training ability of brain) of the blind person, we can directly map each pair of information regarding die traversal time and received power of ultrasound received from a particular direction to a suitable vibro-tactile actuator present in the vibro-tactile array. This process either may follow a predefined lookup table or an inbuilt artificial neural network. NOTE: The number of transmission and receiving channel can be modified according to the specific requirement. c) Vibro-tactile stimulation block The most important aspect regarding the vibro-tactile array we have already designed is mat it doesn't vibrate by its own. We need to activate them by the proper electrical signal having desired frequency. These actuators are operated on 9 to 15 volt (peak to peak) square-wave, so we need to convert each pixel- information of the input image in to corresponding frequency to activate the corresponding actuator. Now to go farther we can consider the output of the signal conditioning block as an image of (n X m), where the possible gray-level value of each pixel relates the information of the obstacle distribution in a specific direction. Fig-4 shows the complete block diagram of the driver system. The core of the actuator driver circuit consists of the three main building blocks 'pixel address and information supply block', 'information to frequency conversion block' and "power amplification block'. As the name suggests the "pixel address and information supply block' supplies the pixel gray-level value (4-bit) sequentially corresponding to their address. Basically the main purpose of this block is to refresh the input informations of the array of information to frequency converters present in the 'information to frequency conversion block". There are individual digitally programmable frequency generators corresponding to each actuator present in the vibro-tactile array. The 'power amplification block' consists of an array of low power darlington-pairs to supply the required current to drive the actuators. Fig-5 shows the basic block diagram of the core of the driver circuit. The tactile actuator array vibrates at frequency up to 100 Hz, delivering significant mechanical energy perceptible to our skin. But to utilize the most sensitive frequency range of the tactile stimulation on our skin, we are selecting the 16 different frequency levels in a logarithmic manner. Table-1 Shows the detail of it. Power consumption of the actuator is another point to be considered while designing a battery operated vibro- tactile actuator array. In our array each actuator consumes 12mA current while operated under 12 Volt. Frequencies for 16 level excitation Frequencies excitation for 6 level Frequencies for 4 level excitation Excitation Level Frequency Excitation Level Frequency Excitation Level Frequency Excitation Level Frequency 1 1.995 Hz 9 12.589 Hz 1 2Hz 1 2Hz 2 2.511Hz 10 15.848 Hz 2 4Hz 2 4Hz 3 3.162 Hz 11 19.952 Hz 3 8Hz 3 8 Hz 4 3.981 Hz 12 25.118 Hz 4 16 Hz 4 16 Hz 5 5.011Hz 13 31.622 Hz 5 32 Hz 6 6.309 Hz 14 39.810 Hz 6 64 Hz 7 7.943 Hz 15 50.118 Hz 8 10.000 Hz 16 63.095 Hz Table-1: Excitation frequency levels Now considering an array of 8X4 =32, 1632 = 2128 is the number of patterns that can be implemented, which is equivalent to a monochrome image having 128 pixel (e.g., 8X 16-monochrome image). But to give the blind user a sense of gray-scale the frequency information of the tactile stimulation is reserved for the information of intensity present in gray-scale image. Distance between the two actuator is another most important design aspect here considering the tactile sensory unit of the skin arm or forearm we have set it around 2cm so that each actuator can be easily and distinctly perceptible. Now the user can be easily trained to percept the pixel informations using their skin of forearm or arm. So using this technique the blind people can easily comprehend a gray-scale image, which is representing the obstacle distribution detected by the data acquisition and processing block. A vibrotactile visual system for the visually impaired people, based on a novel method of echolocation also useful for the other intelligent devices where understanding of obstacle distribution in the surrounding environment is necessary is disclosed. One of the typical implementation of this system comprises an array of ultrasonic transceivers directed in different direction (5, 14), an array ofvibrotactile actuators (18) and electronic circuit module. As this is does not require any bulky laptop or computer for its functionality it is quite lightweight and low cost. This device having a vibrotactile visual interface to the blind person creates very less audio interference with his hearing capability. User also may use it as a supportive traveling aid while using their conventional traveling to get trained on this new system. After getting trained and confidence on this system he may completely rely on this system for his traveling and normal activities. As for using this system users need not to hold anything on his hand they can use their hand for their desired activities.

Full Text

Array based tactile vision for the blind people using multi-directional
ultrasonic scanning of environment
This invention relates to improvement of the devices used as electronics traveling aids for blind people.
Till date electronic traveling aids for the blind people is not so much popular among the blind people
specially in the developing countries like ours, because of their higher cost and the heavier weight So most
often they are traditionally relying on the simple or folding cane as a mobility aid. But such cane can't give them
the entire morphology of the environment in front of them.
There are plenty of traveling aids already designed for the blind people some of them are electronic
specifically utilizing the LASER or the ultrasound, most of mem are stimulating the blind person raising some
alarm, may be audio or tactile depending on the presence of the obstacle in front of him. Some of mobility aids
are delivering continuous audio stimulation to the blind person, which also represents a gross environment in
front of the blind person. But this system highly interferes with the input of the most vital sensory system for the
blind people - hearing capability and at the same time it requires a prolong training to understand the pattern of
the audio stimulation to correlate the obstacle distribution in the environment. Some mobile computer (or simple
laptop) based systems are readily available with the capability of speech-alarm but their cost and weight is very
high.
Another major problem in the detection of the obstacle and its distribution is to identify die rank of the
obstacles i.e., either a direction of the environment is congested with obstacles or the direction has some partial
obstacle. So we need to assign different weight on different type obstacle distribution rather than on an
individual obstacle.
In this project we have developed an ultrasonic tactile visual system with the capability of
interpretation of obstacle distribution in the environment to act as a mobility aid for blind. We had set our goal
to make this system light-weight and low-cost (raw material cost at the development stage is around Rs.5000),
so that it can be used widely for the rehabilitation purpose for the blind people. So we have used the
microcontroller based DSP, instead of using a bulky laptop in the design of the digital electronic circuitry and
embedded software. To interface this system with a blind person we are incorporating array based vibro-tactile
vision. It enables him to enjoy lesser audio interference to their normal hearing capability. We have designed
our system such that the blind person needs not to hold the device in his hand, so that he can utilize his hand for
other activities. Most of all we have made the instrument and its operating procedure such that any blind person
can utilize it without any long term training at least for the localization of die distinct obstacle, whereas a proper
training enables him to understand the morphology and distribution of the obstacles surrounding him.
In addition to the above die data acquisition part used in the above mentioned system is also quiet
useful for position localization of the mobile robots, where we need to implement a little bit different type of
neural-network in the DSP chip. This feature will also enable our system to raise some audio-alarm to the blind
person.
Brief description of the figures:
Fig-l - Fig-5 used for the explanation of the working principle of our system.
Fig-6 shows the schematic of data acquisition block.
Fig-7 shows me schematic of synchronization and processing block.
Fig-8 shows the schematic of vibro-tactile actuator driver block.
Fig-9 shows the PCB layouts of the data acquisition and processing block used in this system.
Fig-10 & Fig-l 1 shows the model of vibro-tactile array and its driver circuit layout designed by us.
Detail Description of the above system:
a) Transmission and reception block:
We know that the continuous transmission of ultra sound creates standing wave by me superposition of me
transmitted beam and the reflected beam, so continuous transmission is not suitable for assessment of the power
of reflected beam. For this reason pulsed transmission of ultrasound is one of the easiest mean for assessing die
power of the reflected beam and the time required to turn-back depending on die distance of the reflecting
obstacle. The following scheme shows die scheme for die distance assessment based on ultrasound traversal

time.
The system we have designed also processes the attenuation of the received ultrasound pulse to assess the
distance of the obstacle. It has 6 (expendable up to 8 or may be reduced as per the specific requirement)
transmission channels directed in a fashion shown in Fig-1 which activates in a pattern of TDM. Here to
enhance the power and efficiency we have designed sinusoidal burst or pulse generating circuit along with a
matching step-up transformer.

Fig-1: Directivity of the transducers mounted in the head band
So we can see that the six transmitters are directed in six different directions for transmitting ultrasound
pulses one by one. For each transmitted ultrasound pulse six detectors receives reflected ultrasound pulses of
different intensities and at different times. These 6 X 6 = 36 intensity & time pairs bear impotent information
regarding the locations of the obstacle in the environment of the ultrasonic visual system.
Now to process the acquired information we have used a PIC series microcontroller - PIC16F877 or
PIC18F452 along with a 16-bit DSP chip - dsPIC30F2011 or higher (optional for inbuilt decision making
feature especially for mobile robots), which not only keeps the synchronization among the various
functionalities of the visual system but also extracts the conclusion from the obtained data. The blind person is
continuously informed about the acquired information by utilizing vibro-tactile vision. When ever the client
sends a requests to this system to know the distribution of obstacles by specifying a directions of their own
interest, the inbuilt decision will deliver the information it him (optional).
b) Signal conditioning and svnchrinization block
The technique we have used here to transmit sequential ultrasonic pulse or burst is basically consisting of
three fundamental building blocks. First of all we have designed a 40 KHz sine-wave generator using the IC
XR2206. Then it is sequentially demultiplexed in 6 different channels of ultrasonic transmitter block. Here each
channel of the demultiplexer is controlled by the control block such that, only up to 16 cycles (programmable
according to the need) of the 40 KHz signal passes to the ultrasonic transmitter block and then the transmission
remains suspend for a duration of 60 mSec for that channel and the next channel is selected. This process is
continued in a cyclic fashion. During this 60 mSec of null transmission period all the 6 (expendable up to 8)
ultrasonic receivers receive the reflected ultrasonic pulses.

Each of the receivers consists of three building locks: the first one is the high pass filter to remove 50 Hz
noise followed by a half wave integrator. The second one is die nearest obstacle determination circuit and both
of them are controlled by third control-block. During the transmission period of 0.2 to 0.4 mSec all the
integrating capacitors are made discharged to keep it ready for the integration during next 60 mSec. At the end
of the 60 mSec of integration all the output voltages of the receiving channels are internally digitized by the
microcontroller. This system is also associated with a temperature sensor to correct the variation of sound-
velocity in air, due to temperature variation.

As in any direction there may lie several partial obstacles so each receiving channel may receive more than
one reflected pulse for a single pulse transmitted from any one of the receivers. For this reason each channel is
associated with a nearest obstacle monitoring block which interrupts the microcontroller with a pre-assigned
priority, whenever the first reflected pulse is received in that channel. Now the microcontroller determines the
time required to obtain the ultrasonic pulse reflected from the nearest obstacle using an internal 16-bit timer. So
at the end of the 61 mSec (approx) we are accompanied with the (6 X 6 X 2 = 72) number of 16-bit data. Now if
we are going to utilize only the neural-network (training ability of brain) of the blind person, we can directly
map each pair of information regarding die traversal time and received power of ultrasound received from a
particular direction to a suitable vibro-tactile actuator present in the vibro-tactile array. This process either may
follow a predefined lookup table or an inbuilt artificial neural network.
NOTE: The number of transmission and receiving channel can be modified according to the specific
requirement.
c) Vibro-tactile stimulation block
The most important aspect regarding the vibro-tactile array we have already designed is mat it
doesn't vibrate by its own. We need to activate them by the proper electrical signal having desired frequency.
These actuators are operated on 9 to 15 volt (peak to peak) square-wave, so we need to convert each pixel-
information of the input image in to corresponding frequency to activate the corresponding actuator. Now to go
farther we can consider the output of the signal conditioning block as an image of (n X m), where the possible
gray-level value of each pixel relates the information of the obstacle distribution in a specific direction. Fig-4
shows the complete block diagram of the driver system.

The core of the actuator driver circuit consists of the three main building blocks 'pixel address and
information supply block', 'information to frequency conversion block' and "power amplification block'. As the
name suggests the "pixel address and information supply block' supplies the pixel gray-level value (4-bit)
sequentially corresponding to their address. Basically the main purpose of this block is to refresh the input
informations of the array of information to frequency converters present in the 'information to frequency
conversion block". There are individual digitally programmable frequency generators corresponding to each

actuator present in the vibro-tactile array. The 'power amplification block' consists of an array of low power
darlington-pairs to supply the required current to drive the actuators. Fig-5 shows the basic block diagram of the
core of the driver circuit.
The tactile actuator array vibrates at frequency up to 100 Hz, delivering significant mechanical
energy perceptible to our skin. But to utilize the most sensitive frequency range of the tactile stimulation on our
skin, we are selecting the 16 different frequency levels in a logarithmic manner. Table-1 Shows the detail of it.
Power consumption of the actuator is another point to be considered while designing a battery operated vibro-
tactile actuator array. In our array each actuator consumes 12mA current while operated under 12 Volt.

Frequencies for 16 level excitation Frequencies
excitation for 6 level Frequencies for 4
level excitation
Excitation
Level Frequency Excitation
Level Frequency Excitation
Level Frequency Excitation
Level Frequency
1 1.995 Hz 9 12.589 Hz 1 2Hz 1 2Hz
2 2.511Hz 10 15.848 Hz 2 4Hz 2 4Hz
3 3.162 Hz 11 19.952 Hz 3 8Hz 3 8 Hz
4 3.981 Hz 12 25.118 Hz 4 16 Hz 4 16 Hz
5 5.011Hz 13 31.622 Hz 5 32 Hz
6 6.309 Hz 14 39.810 Hz 6 64 Hz
7 7.943 Hz 15 50.118 Hz
8 10.000 Hz 16 63.095 Hz
Table-1: Excitation frequency levels
Now considering an array of 8X4 =32, 1632 = 2128 is the number of patterns that can be implemented, which is
equivalent to a monochrome image having 128 pixel (e.g., 8X 16-monochrome image). But to give the blind user
a sense of gray-scale the frequency information of the tactile stimulation is reserved for the information of
intensity present in gray-scale image. Distance between the two actuator is another most important design aspect
here considering the tactile sensory unit of the skin arm or forearm we have set it around 2cm so that each
actuator can be easily and distinctly perceptible. Now the user can be easily trained to percept the pixel
informations using their skin of forearm or arm. So using this technique the blind people can easily comprehend
a gray-scale image, which is representing the obstacle distribution detected by the data acquisition and
processing block.

A vibrotactile visual system for the visually impaired people, based on a novel method of echolocation also useful for the other intelligent devices where understanding of obstacle distribution in the surrounding environment is necessary is disclosed. One of the typical implementation of this system comprises an array of ultrasonic transceivers directed in different direction (5, 14), an array ofvibrotactile actuators (18) and electronic circuit module. As this is does not require any bulky laptop or computer for its functionality it is quite lightweight and low cost. This device having a vibrotactile visual interface to the blind person creates very less audio interference with his hearing capability. User also may use it as a supportive traveling aid while using their conventional traveling to get trained on this new system. After getting trained and confidence on this system he may completely rely on this system for his traveling and normal activities. As for using this system users need not to hold anything on his hand they can use their hand for their desired activities.

Documents:

http://ipindiaonline.gov.in/patentsearch/GrantedSearch/viewdoc.aspx?id=CJ3Qo5U0GqoGLhr3Bbk/Fg==&loc=wDBSZCsAt7zoiVrqcFJsRw==

« Previous Patent

Next Patent »

Patent Number

271918

Indian Patent Application Number

332/KOL/2008

PG Journal Number

11/2016

Publication Date

11-Mar-2016

Grant Date

10-Mar-2016

Date of Filing

22-Feb-2008

Name of Patentee

TIBAREWALA DEWAKI NANDAN

Applicant Address

PROFESSOR & DIRECTOR SCHOOL OF BIO-SC & ENGINEERIG, JADAVPUR UNIVERSITY, PO- JADAVPUR UNIVERSITY, KOLKATA

Inventors:

#	Inventor's Name	Inventor's Address
1	BISWAS ABHIJT	VILL:- RAMMOHON PALLY, POST:- SHYAMNAGAR, DIST:- 24-PGS(N), PIN:- 743127
2	BISWAS SUSHANTA	VILL:- KANTHAL BAGAN POST:- DHUBULIA DIST:- NADIA PIN:- 741139
3	TIBAREWALA DEWAKI NANDAN	PROFESSOR & DIRECTOR SCHOOL OF BIO-SC & ENGINEERIG, JADAVPUR UNIVERSITY, PO- JADAVPUR UNIVERSITY, KOLKATA-700032

PCT International Classification Number

A63F 13/00

PCT International Application Number

N/A

PCT International Filing date

PCT Conventions:

#	PCT Application Number	Date of Convention	Priority Country
1			NA