How Sensor-Packed Smartphones Can Read Your Mood, Guard Your Data — and Wreak Havoc in the Wrong Hands

Smartphones can already do pretty much everything, right? Actually, UAB computer scientists have a few more ideas.

They’re tapping into the accelerometers, proximity sensors and other environment-aware chips packed into modern phones to help users stay safe — and keep ahead of the bad guys.

Here are seven innovations that could be coming soon to your favorite device.

1. Watching your back

Most of us are very protective of our phones. Ragib Hasan, Ph.D., an assistant professor in the UAB College of Arts and Sciences Department of Computer and Information Sciences and director of SECRETLab, wants them to return the favor. He is developing software to turn a phone into a digital wingman, using information from its camera, microphone, accelerometer and other sensors to gauge a user’s attentiveness and respond appropriately. When it detects that a person is driving, for example, it could silence all but the most important alerts. If it decides from the way that you’re walking and talking that you are drunk, it could prevent you from making bank transactions. Hasan’s code will also save important security warnings for times when you are alert, rather than groggy from sleep.

The project builds on a study by Munirul Haque, Ph.D., who recently completed a postdoctoral fellowship in Hasan’s lab, and collaborators at Marquette University. The researchers found that a phone can do a remarkably good job at sensing mood. They parsed camera images to read facial expressions and accelerometer data to judge energy expenditure (anxious people tend to pace; inactivity is often a signal of depression). Their system was able to recognize six different “affective states”: anger, disgust, fear, happiness, sadness and surprise.

Learn more: Read the paper “In Situ Affect Detection in Mobile Devices: A Multimodal Approach for Advertisement Using Social Network”

2. Learning your style

You may be only one of millions of people with an iPhone, but the way you hold your phone — and take pictures and send text messages — may be unique. Nitesh Saxena, Ph.D., an associate professor in the CIS department and director of the SPIES lab, is a pioneer in “behavioral biometrics” security research. He’s pulling together data from accelerometers, gyroscopes and proximity sensors to chart the characteristic gestures a user makes when answering a call or snapping a selfie. Once his software learns your moves, it could unlock your phone automatically — and freeze when it detects that it is in the wrong hands. A system that taps into user interactions with multiple connected devices, such as Google Glass or the new Apple Watch, would be even more secure, Saxena says.

Learn more: Read the Mix feature Swagger security: How your smartphone style could keep your digital assets safe

Read more: How Sensor-Packed Smartphones Can Read Your Mood, Guard Your Data — and Wreak Havoc in the Wrong Hands