People looking to a popular online symptom checker for answers about eye problems often get a wrong diagnosis, a small study suggests.

Symptom checkers are all over the web, often using algorithms to deliver diagnoses even as they post disclaimers advising against using these tools for medical advice. While some studies have already documented many ways that symptom checkers can mislead patients, less is known about how well they work specifically for eye conditions, researchers note in JAMA Ophthalmology.

For the current study, researchers tested how often an online symptom checker offered by WebMD generated the correct diagnosis for 42 clinical vignettes similar to situations that patients might describe to doctors in person. They wanted to see how often the symptom checker got the correct diagnosis or at least generated the right diagnosis as one of the top three possible answers. The tests were conducted in late 2017.

Overall, the symptom checker nailed the diagnosis for just 11 of the 42 cases, or only 26 percent of the time. And the correct diagnosis was among the top three possible diagnosis in just 16 cases, or 40 percent of the time.

“Ultimately, what’s at stake here is the patient’s vision and peace of mind,” said lead study author Dr. Carl Shen of McMaster University in Hamilton, Canada.

“If patients receive the wrong diagnosis or are told to not urgently seek medical care when they otherwise should have, they may run the risk of losing their eyesight, sometimes permanently,” Shen said by email.

Out of the 42 vignettes in the study, 18 were “emergent” conditions that needed immediate care and 24 were “nonemergent.” On average, the vignettes included three to four symptoms, and many had at least one symptom that didn’t directly involve the eye.

The symptom checker got the urgency level of the situation right in just seven of the emergency cases, or only 39 percent of the time. It correctly categorized most of the non-emergency cases as situations that didn’t need immediate care.

When the symptom checker generated a long list of possible diagnoses, the correct one was usually 4 or 5 places down the list, on average, not in the top three.

The most common diagnosis made by the symptom checker was nearsightedness, which was given in nine cases.

It’s unclear how closely the vignettes used in the study might match what patients would enter in an online symptom checker in real life. The study also wasn’t designed to determine whether or how these online tools might influence patients’ decisions about seeking medical care.

“The version of the WebMD Symptom Checker used in this study represents the older version of the online tool, which did not use the latest algorithm and technology that drives the current version,” Dr. Michael Smith, chief medical director for WebMD, said in an emailed statement.

“Symptom checkers are intended for informational purposes only,” Smith said. “They do not provide medical advice, and they are not a substitute for professional medical diagnosis.”

Still, the study highlights that patients need to proceed with caution when consulting “Dr. Google,” said the author of an accompanying editorial, Dr. Rahul Khurana of Northern California Retina Vitreous Associates in Mountain View, California.

“This is the first study to look at ocular online symptom checkers, and it clearly shows that they are not accurate,” Khurana said by email.

“The correct diagnosis is a combination of clinical history, physical exam, and physician input to generate the correct diagnosis,” Khurana added. Online symptom checkers “could be a starting point, but they are very inaccurate so I would be cautious about them.”