Robotic walking stick to make navigation easier for the blind, visually impaired

A trio of engineers has developed a smart cane that provides navigation assistance with warning vibrations and spoken directions, enhancing the independence of people with blindness or visual impairments by helping them find a seat in a public space or the right products in a grocery store, among others.

The team’s smart cane looks like ordinary ones that can be bought at local department stores but with a few additions: a camera, sensors, and vibrational motors powered by a backpack-mounted laptop. This system leverages technology to perform 1) simultaneous localization, mapping (SLAM), and object localization; 2) goal disambiguation and scoring; and 3) path planning and guidance.

“Goal-based navigation in public places is critical for independent mobility and for breaking barriers that exist for blind or visually impaired people in a sight-centric society,” said the engineers from the University of Colorado Boulder in Boulder, Colorado, US, led by Shivendra Agrawal, a doctoral student in the Department of Computer Science.

“Through this work we present a proof-of-concept system that autonomously leverages goal-based navigation assistance and perception to identify socially preferred seats and safely guide its user towards them in unknown indoor environments,” they added.

Best place to sit

In a pilot user study, Agrawal and colleagues conducted experiments to test the usefulness of the smart cane in terms of locating a seat in a café. The team set up a makeshift café that had several chairs, patrons, and a few obstacles.

“Imagine you’re in a café. You don’t want to sit just anywhere. You usually take a seat close to the walls to preserve your privacy, and you usually don't like to sit face-to-face with a stranger,” Agrawal pointed out, adding that this concern is a priority of people who are blind or visually impaired.

Then, wearing blindfolds, participants without visual impairments picked up their smart cane and strapped on a backpack that held the laptop that powered the walking device. The device helped the participants navigate the café and successfully find the right chair in 10 out of 12 trials with varying levels of difficulty. [IEEE Int Conf Intell Robots Syst 2022;doi:10.1109/IROS47612.2022.9981219]

Right grocery item to buy

In another experiment, the team set up a makeshift grocery shelf stocked with multiple varieties of cereal. In line with this, the smart cane was modified for a task that required locating and grabbing things in aisles filled with dozens of products that were comparable in appearance and feel. A database of product photos, including boxes of Honey Nut Cheerios or Apple Jacks, was also created onto a software.

The idea was to allow the walking stick to scan the shelf, evaluate the available products, and pinpoint the specific item that the user wanted to buy.

“It assigns a score to the objects present, selecting what is the most likely product. Then, the system issues commands like ‘move a little bit to your left’,” Agrawal said.

A powerful enabler

The results of the second experiment have yet to be released, but Agrawal and colleagues believe that their smart cane would increase its user’s confidence in independent navigation and enable them to freely engage in socially dynamic situations like seat choice and grocery shopping.

It will take some time before the smart cane becomes available commercially and reaches the hands of blind and visually impaired individuals, they said. In the meantime, the team is looking to make the device more compact, such that it can run with a standard smartphone attached to a cane rather than with a laptop in a backpack.

“Our aim is to make this technology mature but also attract other researchers into this field of assistive robotics,” Agrawal said. “We think assistive robotics has the potential to change the world.”

The study was presented at the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) held in Kyoto, Japan.