No braille? No problem. A new device lets blind people read by popping a miniature camera on their fingertip.
To read printed material, many visually impaired people rely on mobile apps like KNFB Reader that translate text to speech. Snap a picture and the app reads the page aloud. But users sometimes find it difficult to ensure that their photo captures all of the text, and these apps can have trouble parsing a complex layout, such as a newspaper or restaurant menu.
“We want to empower end users to accomplish these activities of daily living through technology,” says Jon Froehlich at the University of Maryland.
Froehlich and his colleagues have developed a device, nicknamed HandSight, that uses a tiny camera originally developed for endoscopies. Measuring just one millimetre across, the camera sits on the tip of the finger while the rest of the device clasps onto the finger and wrist. As the user follows a line of text with their finger, a nearby computer reads it out. Audio cues or haptic buzzes help the user make their way through the text, for example changing pitch or gently vibrating to help nudge their finger into the correct position.
In a study published in October, 19 blind people tried out the technology, spending a couple of hours exploring passages from a school textbook and a magazine-style page. On average, they were able to read between 63 and 81 words per minute and only missed a few words in each passage. The average reading speed for an expert braille reader is around 90 to 115 words per minute, while sighted individuals have an average reading speed around 200 words per minute.
Point and read
Some participants were enthusiastic: “I’m very pleased and excited about the system. I think it could make a great difference in my life,” one told the researchers. Others were less certain, and suggested it was easier to find the next line and read at a steady pace using braille or other technology.
Matthew Janusauskas at the American Foundation for the Blind, a nonprofit based in New York City, thinks the technology could be useful for reading printed materials where the layout affects comprehension, such as a page with multiple columns of text. “What they’re doing looks very interesting,” he says. “The innovation would be in the usability.”
Down the line, the creators of HandSight imagine a much more dynamic, smartwatch-like device that blind people could use not only to read text but also to discern other visual characteristics, like colours and patterns.
“They’re already using fingers all of the time to explore the physical world,” says Froehlich. With HandSight, his team hopes to “augment the fingers with vision to allow blind or severely visually impaired users to get a sense of the non-tactile world”.
Journal reference: ACM Transactions on Accessible Computing, DOI: 10.1145/2914793