Federal and State Websites Flunk COVID-19 Reading-Level Review

Online pages used long sentences, complex wording when sharing health information.

Information about COVID-19 offered by the U.S. Centers for Disease Control and Prevention, the White House, and all 50 state health departments failed to meet recommendations for communicating with the public, according to a Dartmouth study. In some cases, the CDC failed to meet its own recommendations.

"How public health information is presented can influence understanding of medical recommendations," says Joseph Dexter, a fellow at Dartmouth's Neukom Institute for Computational Science and senior author of the study. "During a pandemic it is vital that potentially lifesaving guidance be accessible to all audiences."

A review of written web content during the onset of COVID-19 demonstrates that official information about the virus might have been too complex for a general audience, according to the researchers. On average, government sources communicated about three grades higher than the reading level recommended by existing guidelines for clear communication, they found.

The research, which also analyzed international health web pages, appears as a research letter in JAMA Network Open, a medical journal published by the American Medical Association.

The U.S. Plain Writing Act of 2010 requires that "federal agencies use clear government communication that the public can understand and use." The CDC, the American Medical Association, and the National Institutes of Health all recommend that medical information for the public should be written at no higher than an eighth-grade reading level.

After studying 137 web pages from federal and state sources, the team found that communications about COVID-19 averaged just over an 11th-grade reading level. According to the study, information shared by the CDC and other U.S. sources during the onset of COVID-19 exceeded recommendations for the number of words in a sentence, word size, and the use of difficult terms related to public health.

Even though states are not required to follow the federal regulations, they might still benefit from complying with the guidelines, says Dexter. All 50 U.S. states provided information above the eighth-grade level. Nine of the 10 states with the highest illiteracy rates had websites written above a 10th-grade level.

Previous research has found that 12 percent of American adults exhibit proficient health literacy.

"The differences between eighth-grade and 11th-grade reading levels are crucial. Text written at a higher grade level can place greater demands on the reader and cause people to miss key information," says Dexter.

According to the study, exceeding readability standards may have a greater effect on vulnerable communities with lower health literacy, potentially worsening the impact of the pandemic.

"Information about COVID-19 can be complex, contradictory, and sometimes false. It is important that members of the public be able to use public health recommendations from trustworthy sources and not have to turn to less reliable outlets," says Dexter.

To assess the readability of information related to COVID-19, the research team analyzed web sites during the critical beginning stages of the pandemic in early April. Pages consulted were meant for general audiences and included lists of frequently asked questions and fact sheets.

Beyond the U.S., the researchers reviewed 18 international websites, including three public health agencies and 15 official government sites.

Almost all of the international pages scored above the eighth-grade level by five different assessments. All of the pages, including those from the World Health Organization, exceeded the U.S. national eighth-grade guideline by at least one measure.

"WHO coordinates a major effort to redirect anyone searching for information about COVID-19 to reliable sources," says Vishala Mishra, a co-author of the paper and a researcher at Madras Medical College. "Therefore, it becomes especially relevant for governments and health agencies, such as WHO and CDC, to provide more accessible health information that matches the public's health literacy."

Web pages containing COVID-19 information were judged using standard readability formulas that estimate text difficulty based on features such as word and sentence length. The reading-level predictions of these formulas were assessed against health literacy guidelines based on federal regulations.

For the latest information on Dartmouth's response to the pandemic visit the COVID-19 website.

David Hirsch can be reached at david.s.hirsch@dartmouth.edu