Crowdsourcing with Mobile Apps Brings ‘Big Data’ To Psychological Research

Sample images from Airport Scanner: the left image contains one target (hand grenade), the middle contains two identical targets (two exemplars of the dynamite stick target type), and the right image contains two different target types (derringer, gasoline can). Airport Scanner images appear with permission from Kedlin Co. Copyright 2014 by the Kedlin Company.

A fast-paced game app where players pretend they are baggage screening officers operating airport x-ray scanners has provided researchers with billions of pieces of data in record time, according to an article published by the American Psychological Association.

“Mobile devices offer researchers an exciting new means to crowdsource an experiment using games that are actually tests of cognition or other brain functions,” said Stephen R. Mitroff, PhD, of Duke University, lead author of the report, published in APA’s Journal of Experimental Psychology: Human Perception and Performance^®. “Questions that could have taken decades to answer in a laboratory setting, or that could not be realistically answered in a lab, can be examined using big data gathered in a relatively short time.”

To demonstrate the potential of mobile technology to gather data, the researchers partnered with Kedlin Co., the developer of the popular mobile app game Airport Scanner, which challenges players to identify illegal items in luggage passing through an airport x-ray scanner. Players view one bag at a time and tap their touchscreen to identify banned items. The game includes a logbook of illegal items (e.g., guns, hand grenades, switchblades) and legal items (e.g., ear phones, clothing), and the list expands from a handful of possible target items to hundreds as players progress through the game.

Between January 2013 and November 2014, the game provided anonymous data from more than 2 billion trials from over 7 million mobile devices that Mitroff and his research team at the Duke Center for Cognitive Neuroscience have been analyzing, the report said. Players consent to the data collection when installing the game on their smartphone or other hand-held device.

One research question explored through the Airport Scanner game was examining players’ ability to spot items that occurred on screen very rarely, or in less than 0.1 percent of all bags.

“When a target appears only 0.1 percent of the time, you need 1,000 trials to get just a single case of that target. Too many trials would be needed to realistically assess detection of that particular target in a laboratory setting,” Mitroff said. “With the large dataset, we were able to look at hundreds of cases for each of the nearly 30 rarest targets.”

Read more: Crowdsourcing with Mobile Apps Brings ‘Big Data’ To Psychological Research