CORA study examines students’ ability to critically assess information from the Internet and from social media

The Internet and social media are among the most frequently used sources of information today. Students, too, often prefer online information rather than traditional teaching materials provided by universities. According to a study conducted by Johannes Gutenberg University Mainz (JGU) and Goethe University Frankfurt, students struggle to critically assess information from the Internet and are often influenced by unreliable sources. In this study, students from various disciplines such as medicine and economics took part in an online test, the Critical Online Reasoning Assessment (CORA). “Unfortunately, it is becoming evident that a large proportion of students are tempted to use irrelevant and unreliable information from the Internet when solving the CORA tasks,” reported Professor Olga Zlatkin-Troitschanskaia from JGU. The study was carried out as part of the Rhine-Main Universities (RMU) alliance.

Critical evaluation of online information and online sources are particularly important today

Learning using the Internet offers many opportunities, but it also entails risks. It has become evident that not only “fake news” but also “fake science” with scientifically incorrect information is being spread on the Internet. This problem becomes particularly apparent in the context of controversially discussed social issues such as the current corona crisis, but it actually goes much deeper. “Having a critical attitude alone is not enough. Instead, Internet users need skills that enable them to distinguish reliable from incorrect and manipulative information. It is therefore particularly important for students to question and critically examine online information so they can build their own knowledge and expertise on reliable information,” stated Zlatkin-Troitschanskaia.

To investigate how students deal with online information, Professor Olga Zlatkin-Troitschanskaia and her team have developed a new test based on the Civic Online Reasoning (COR) assessment developed by Stanford University. During the assessment, the test takers are presented with short tasks. They are asked to freely browse the Internet, focusing on relevant and reliable information that will help them to solve the tasks within the relatively short time frame of ten minutes, and to justify their solutions using arguments from the online information they used.

CORA testing requires complex and extensive analysis

The analysis of the results is based on the participants’ responses to the tasks. In addition, their web search activity while solving the tasks is recorded to examine their strengths and weaknesses in dealing with online information in more detail. “We can see which websites the students accessed during their research and which information they used. Analyzing the entire process requires complex analyses and is very time-consuming,” said Zlatkin-Troitschanskaia. The assessments have so far been carried out in two German federal states. To date, 160 students from different disciplines have been assessed; the majority of the participants studied medicine or economics and were in their first or second semester.

Critical online reasoning skills should be specifically promoted in higher education

The results are striking: almost all test participants had difficulties solving the tasks. On a scale of 0 to 2 points per task, the students scored only 0.75 points on average, with the results ranging from 0.50 to 1.38 points. “The majority of the students did not use any scientific sources at all,” said Zlatkin-Troitschanskaia, pointing out that no domain-specific knowledge was required to solve the CORA tasks. “We are always testing new groups of students, and the assessment has also been continued as a longitudinal study. Since we first started conducting these assessments two years ago, the results are always similar: the students tend to achieve low scores”. However, students in higher semesters perform slightly better than students in their first year of study. Critical online reasoning skills could therefore be promoted during the course of studies. In the United States, a significant increase in these kinds of skills was observed only a few weeks after implementing newly developed training approaches.

The study shows that most students do not succeed in correctly evaluating online sources in the given time and in using relevant information from reliable sources on the Internet to solve the tasks. “As we know from other studies, students are certainly able to adequately judge the reliability of well-known media portals and Internet sources. We could build on this fact and foster the skills required to critically evaluate new sources and online information and to use the Internet in a reflected manner to generate warranted knowledge,” concluded Professor Olga Zlatkin-Troitschanskaia.

In research on this topic, skills related to critically dealing with online information and digital sources are regarded as an essential prerequisite for learning in the 21st century. However, there are still very few training approaches and assessments available for students to foster these skills, especially online. “The RMU study is still in the early stages of development. We have only just developed the first test of this kind in Germany,” Zlatkin-Troitschanskaia pointed out. “We are currently in the process of developing teaching/learning materials and training courses and of testing their effectiveness. The analysis of the processing will be particularly useful when it comes to offering students targeted support in the future.