Be My AI Replies Run 10x Longer for Blind Users

Be My AI, an app that describes photos for blind people, gave answers more than ten times longer than the questions users asked, according to a University of Maryland study of 11 blind participants. The researchers say the app often helped, but that users had to do too much work to get the one detail they needed.

The study was presented at the CHI human-computer interaction conference earlier this year and described by the university on 29 October. “We find that these systems are getting better at generating descriptions, but without effectively capturing user intent, that capability often translates into lengthy responses rather than useful answers,” said co-author Hernisa Kacorri, an associate professor in the university’s College of Information.

In brief

11 blind participants used Be My AI in a lab and then for 10 days at home.
Questions were typically 10 to 20 words; replies were more than ten times longer.
The researchers call for verbosity controls, better camera guidance and ways to check high-stakes answers.

What the Be My AI study found

Be My AI is part of Be My Eyes, which has linked blind and low-vision people with sighted volunteers since 2015. The AI feature, added in 2023, answers questions about a photo straight away.

Participants practised with it in a lab, used it in daily life for 10 days and then discussed what worked. The team analysed 418 conversations. Exchanges sometimes stretched to 21 back-and-forths, and because blind users hear the replies read aloud, they had to sit through long descriptions to find what mattered.

Part of the problem is the app’s design. Be My AI starts every conversation with a full description of the photo, and, according to the paper, users could not attach a question to the first image. More than half the participants called this a serious problem. “It gave me a bunch of information that I did not want,” said one user who only wanted to find her keys.

The paper also says the system “lacked verbosity controls, was limited in estimating distance in space and time, relied on inaccessible image framing, and offered little to no camera guidance”. Even so, users considered 83% of their interactions successful.

How blind users worked around long answers

Participants found their own fixes. Some told the app why they were asking. One said stating a goal lets you “narrow the lane dramatically … and you’ll get to the finish line much more quickly.” Another broke the job of checking bread dough into separate questions about size, surface, air bubbles and flour. It took longer, she said, but it was “the only way to make sure that it was 100% correct.”

One participant, getting ready to go out, wanted a quicker route: “If you had a hurry button and I was in a hurry, that probably would be OK.”

Why the burden should shift from users to app makers

The participants were inventive, but the researchers argue that should not be the answer. A complex prompting strategy, they told the university, “is a workaround for an existing limitation of the technology.” Yuehan Qiao, a PhD student on the team, said that in time-sensitive situations the burden “can not only delay action but also create safety concerns.”

The fixes they propose are specific: a control to switch between short and detailed answers, directions described from the user’s position rather than the photo’s frame, active help with pointing the camera, and a way to double-check answers when the stakes are high. Some of these are product choices rather than model upgrades, such as a short-answer setting or letting users ask their question with the first photo. Those could arrive without waiting for a better model.

The evidence has clear limits. Eleven people over ten days is a small qualitative sample, and the study tested one app at one point in its development. The app also did not remember preferences between sessions, which shaped what participants tried. It shows what needs fixing, not how common each problem is.

Assistive AI is spreading quickly, from AI glasses that describe paintings in a Milan gallery to sign language interpreters on FaceTime, and TSN’s tech for good roundup tracks similar work. The team has also released its conversation data publicly. The next sign of impact will be whether Be My Eyes or rival apps add a short-answer mode.