A Phone App Could Help Blind and Low-Vision People Walk Outdoors More Independently

For people who are blind or have low vision, an ordinary walk outside means finding the curb cut, staying on the sidewalk where the path forks, and getting around the sign post or the open car door before reaching it. The usual tools each cover part of that job. A white cane finds what is underfoot. Guide dogs and electronic travel aids, the researchers note, can be expensive or hard to get. Map apps give directions, but according to Harvard, phone GPS on its own is not accurate enough to use safely for that kind of walking.

A team at Harvard has built a phone app that tries to do all of it at once. It is called Mobilio, and in tests with 14 blind and low-vision volunteers, people finished an outdoor route faster and had fewer contacts with obstacles than when they used Google Maps and a white cane. The work, led by Ph.D. student Raymond Liu and assistant professor Patrick Slade, was published on August 24, 2026 in Nature Biomedical Engineering.

Starting with what people asked for

Before building the app, the team surveyed 112 people with blindness or visual impairment. Their answer was clear: a good outdoor aid needs turn-by-turn directions, guidance that keeps you on the path, and a way to detect and get around obstacles. Existing tools, the authors write, often lack one or more of those.

What the machine learning does

The core of the app is a computer vision model that looks at the live camera feed and works out where it is safe to walk. Most street-scene datasets are filmed from cars, so standard models tend to misread the things a pedestrian cares about. According to Harvard’s announcement, Liu trained a segmentation model on pedestrian-view images so it could recognize sidewalks, crosswalks and roads during a walk. Harvard says it works alongside the phone’s GPS, motion sensors and, on phones that have one, a LiDAR depth sensor, all running on the phone itself.

“Our system uses the sensors in a smartphone, GPS information, and the actual motion of the person to create what’s essentially a small autonomous vehicle plan on how to navigate them from where they are to where they want to go,” Patrick Slade, Harvard SEAS

Instead of a voice reading out instructions, Mobilio plays beeps that come from the left or the right, steering the walker with a gentle nudge. Harvard says the app measures how closely each person follows the beeps and adjusts the cues in real time to suit them, an approach it calls human-in-the-loop optimization.

How it did with real walkers

The 14 volunteers were recruited through the Carroll Center for the Blind in Newton, Massachusetts. Each walked with Google Maps and a cane, and again with Mobilio and a cane, on a pre-planned outdoor path and on an indoor obstacle course. With Mobilio, they finished the outdoor community path in 13 percent less time. The paper’s abstract reports a 41 percent reduction in contacts with the environment in the same sentence as the community path result; Harvard’s release places that 41 percent reduction on the indoor obstacle course. On reliability, meaning how consistently it got people to the destination, the paper says Mobilio was similar to a human guide. Participants reported that it was easy to use, took little effort and gave intuitive audio cues.

The team has posted navigation code samples and the experiment data from all 14 participants on GitHub, so other groups can check and build on the work.

The person this was built for

Liu has an older brother who is blind, and Harvard says supporting his brother’s independent travel shaped the goals of the work. That is the real measure here: whether someone can walk to a friend’s house, a bus stop or a store on their own, with a phone already in their pocket. Mobilio is not something you can download yet. Harvard says the next step is testing it in a wider range of real settings around Boston, and that the Harvard Grid Accelerator is supporting work to turn it into a future product.