Can ordinary words like "and" or "not" in online reviews provide valuable insights to healthcare providers about their patients' experiences?
A new study reveals they can. The analysis showed that negative reviews of healthcare facilities primarily focused on communication and administrative problems, while positive reviews highlighted reassuring interactions with clinicians and staff. This was discovered by examining specific words used in these reviews and correlating them with positive or negative feedback.
The research was conducted by a team at the Perelman School of Medicine at the University of Pennsylvania and published in JAMA Network Open.
JAMA Network OpenAs many individuals rely on online reviews for various industries, researchers see these reviews as an invaluable resource for understanding true patient sentiments.
"Current methods used by healthcare facilities to measure the patient experience often lag between actual encounters and subsequent improvements," said senior author Anish Agarwal, MD, MPH, deputy director of the Center for Insights to Outcomes. "If real-time data from online reviews could be incorporated into easily accessible tools, health systems could make almost immediate changes to enhance patient experiences."
The researchers analyzed over a million Yelp reviews covering seven years (2017 through 2023) across nearly 139,000 healthcare facilities in the United States. They used a tool called the Differential Language Analysis ToolKit (DLATK) to specifically examine 1- and 2-star (negative) reviews as well as 4- and 5-star (positive) reviews.
Around 46% of all reviews were rated 1- or 2-stars. The presence of the word "not" was most predictive of negative feedback, with payment issues and poor treatment being significant factors tied to such reviews.
The analysis also revealed that words like "told," "said," "call," and "asked" frequently correlated with critical feedback. Prior research by Agarwal had similarly found "told" as the most common term in adverse online hospital reviews.
Other common negative themes included phone conversations, hold times, perceived unfair payment scenarios, and insufficient medical care, although poor treatment was less frequent than expected according to Agarwal.
More than 50% of reviewed texts were positive (4- or 5-stars), with "and" being the top term linked to these favorable assessments. Terms like "great," "friendly," and "kind" also ranked highly in positive reviews, emphasizing the importance of reassuring interactions.
The ability of staff members to ease patients' anxieties was highlighted as well, making terms such as "nervous" and "scared" actually correlate with positive experiences.
An analysis noted a decrease in healthcare reviews during the COVID pandemic, but the research team remains optimistic about using these insights for improving care quality.
"These are broader systemic issues, not isolated complaints," said first author Neil Sehgal. "We're keen to understand how factors such as facility size, ownership, or patient volume might influence patient sentiment and communication challenges."
Agarwal is particularly interested in learning more about healthcare providers who consistently receive excellent reviews.
"I'd like to see which hospitals are excelling despite recent years and understand their strategies for high quality and well-regarded care," Agarwal said. "By learning from the best, everyone can improve."