Critical Thinking Skills to Help Students Better Evaluate Scientific Claims

Michelle Joyce doesn’t shy away from politicized science topics such as climate change. In fact, she works to equip seniors at Palmetto Ridge High School in Naples, Florida with the skills to accurately evaluate those topics on their own. Along with teaching chemistry and physics, she offers a class called “thinking skills” where students solve logic and math puzzles while also enhancing their media literacy. Students go beyond just learning about legitimate sources of information on the internet and delve into just how the information is put together in the first place.

But teaching students those critical thinking skills only as they’re about to depart for college can be too little too late.

“It’s a really hard thing to teach within the space of everything else that you need to teach in a classroom,” Joyce said. “It’s crucial that we teach it as early as we can.”

The internet has no shortage of dubious information; and the ability to evaluate health and science claims is a subset of media literacy. With the abundance of health/science content students may only see via social media, kids are ill-equipped to discern hype from real science.

In one recent study by the Stanford History Education Group, 170 high school students were shown a photo of flowers growing fused together and asked if that provided strong evidence on the conditions outside the Fukushima Daiichi nuclear power plant. Students with mastery of media literacy would argue this was not sufficient evidence because there is no information on the source of the photo or where the flowers were photographed. However, less than 20 percent of the students responding made that argument. Nearly 40 percent argued that the picture alone was strong evidence for conditions outside the nuclear plant.

“We are swimming in bullsh-t and lots of different claims about what helps or harms us,” said Dr. Andrew Oxman, director of research at the Norwegian Institute of Public Health. “Everybody needs to figure out which claims are trustworthy.”

START EARLY

Many of these critical thinking concepts are not difficult but need to become habits adopted early in life, which is why Oxman first tried them out in his children’s elementary school classrooms. One way to teach how science is made is to let the children experience and figure it out for themselves.

Oxman gave students a bag of M&Ms and told them that some kids thought the red ones helped them study better but others got stomach aches. He instructed students to evaluate these claims.

“They figured out very quickly, you have to compare like to like,” said Oxman.

The most revealing aspect of this lesson was how quickly students understood the pitfalls of setting up a randomized study. The teacher mentioned they could set random assignments much like they do in gym class where they set up teams by alternating students in line.

“The kids started laughing because they understood right away that doesn’t work,” said Oxman. Students learned they can sabotage randomization in picking teams by setting up a line so they are one student away from their friend.

“In research jargon, we referred to that as ‘concealed allocation’ and it’s a concept that takes time to explain to health professionals but the kids understood it right away intuitively.”

Through this experiment students quickly figured out they had to measure things exactly same. They discovered the flaws of using small samples and being misled by games of chance, he added.

Oxman has since taken this idea of teaching young children concepts of evaluating science to a much larger scale.

Teaching Health Claims

He and a global team of researchers at Informed Health Choices developed a study of some 10,000 Ugandan fifth-graders to see if a simple comic book on evaluating health claims could provide students with the skills to make better choices about their health. The comic book begins by describing how one child -- who has burned his finger -- sticks his wound into dung to heal it. The finger gets infected and he visits Professors “Fair” and “Compare” and begins to learn about how to question and evaluate the health advice he receives.

The workbook had a convincing effect, Oxman said. The students who received the workbook and those who did not receive it were then tested on how to evaluate health claims. Fifty percent more children in the workbook group had a passing score on that critical thinking test. Twenty percent of the students receiving the workbook even showed mastery of the concepts.

As a follow-up to the study, researchers are asking children and adults what they learned and how they’ve used it. Responses so far have been very promising, he added.

One girl talked about going shopping with her mom, who picked up an expensive new brand of toothpaste, but the girl picked up an older brand of toothpaste and found the ingredients were the same. During the pilot studies, Oxman said it was fun to see kids walking out of class talking to each other about claims. Recognizing a claim, and being able to determine if it’s trustworthy is the critical first step to appraising all the claims people hear every day, he said. 

EVALUATING CLAIMS

Palmetto Ridge High School science teacher Michelle Joyce said she uses a process called “claim, evidence and response.”

First students recognize a claim or a hypothesis. Next, they look at evidence: the original data; who calculated the data; where the study was conducted; if the researcher would be inclined to benefit from a certain result; if researchers did multiple trials or tested on many people and more. Finally, students must come up with a response: a determination of the validity of the claim.

Joyce uses resources from a variety of places including Common Sense Education*, a nonprofit that provides free curriculum in media literacy for grades K-12.

When teachers tackle the subject of media literacy they may think about social media etiquette or cyberbullying -- that’s a component of media literacy called digital citizenship. But teaching media literacy can also go into specific domains such as health and science. To understand the science news they see online, kids need to understand basic concepts like sample sizes or what “peer review” means, said Jeffrey Knutson of Common Sense Education. Through this curriculum, students learn how information is created and distributed.

“It gives them an insider’s view of how information we get is created and how we receive it,” said Knutson.

Digital citizenship and media literacy is often taught as something extra and not necessarily embedded in curriculum, said Knutson. However, health or science claims seen online can be easily incorporated into science or health class.

One example Knutson provided was a recent New York Times article about health hazards of chemicals used in packaging such as boxed macaroni and cheese. The article stirred up some controversy because it didn’t offer specifics on what dose of these chemicals can do damage. The study was financed by an environmental advocacy group, not an unbiased source. The Times reported on this study and other journalists reported on it and then reports about the reporting came out, noted Knutson.

“The best thing teachers can do is to use these examples in their class with their students," according to Knutson. "It’s important to model how you would go about reading an article like that.”

BREAKING DOWN THE FACTS

High school science teacher Michelle Joyce says that if teachers start in elementary school, and build on these concepts in middle and high school, “we have significantly more chance over a period of time to build this common sense, this media literacy,” said Joyce.

“If I’m only seeing them in 11^th and 12^th grade, many of their opinions are already formed,” she added.

When Joyce really breaks a scientific topic into its component parts, she can sometimes convince skeptical students.

For instance, students were learning the bleaching of the Great Barrier Reef, which stems from water temperature and salt-level or salinity increases. They researched where the increased salinity or temperature could be coming from, including climate change and waste dumped near the ocean.

One student said, “I can’t believe one degree in temperature makes this much a difference for these animals,” recalled Joyce.

Instead of rehashing the reef data, Joyce brought the conversation back to a different perspective and explained pH levels in the human body. Even a slight change in pH could shut down a person's bodily functions. Suddenly the minor change to water temperature and salinity, which affects pH, didn’t seem so minor.

“I have to bring it back to something they can relate to,” said Joyce.

Joyce can’t go into such detail with every science lesson but she hopes by equipping students with the skills to question what they read, they’ll be able to pursue these questions on their own.

“Teaching them those skills on how to think like a scientist and how to analyze information that they’re receiving is just as important as teaching them to use the periodic table, for example.”

Additional resources: 

Common Sense has several strategies to help debunk false claims, such as those listed on “Turn Students into Fact-Finding Web Detectives.” They also offer digital citizenship and information literacy curriculum.

Health News Review covers often-hyped health stories.

Retraction Watch  highlights retractions of studies and other pitfalls of the peer review system, which rarely gets attention.

*Common Sense Education contributes reviews of educational tools and other content to MindShift.