New app using an ordinary smartphone can tell if an apple has been treated with pesticides and more

• New smartphone app looks inside objects.
• “HawkSpex® mobile” displays constituents.
• Hardware accessories are not required for cameras.

An apple is labeled “organic” but the customer doesn’t know if it really hasn’t been sprayed or, if the seller’s description is to be believed, if a car has never been in an accident. Unverifiable information has to be accepted in many situations. The “HawkSpex® mobile” app from the Fraunhofer Institute for Factory Operation and Automation IFF in Magdeburg will enable consumers to verify such information in the future. The principle: Take out your smartphone, open the app, aim it at the object being scanned – such as an apple – and get the desired information, for instance, whether an apple contains pesticide residues.

Although systems that perform such scans already exist, users usually have to clamp additional parts such as a prism onto the front of the integrated camera. This is costly and impractical and additionally interferes with a smartphone’s design. “What makes our app special is that users don’t need anything for a scan other than the camera already integrated in their smartphones,” says Prof. Udo Seiffert, Expert Group Manager at the Fraunhofer IFF.

No hyperspectral camera required

How did the development engineers working with project manager Dr. Andreas Herzog manage to get by without a prism? Such scans usually require a special hyperspectral camera: It adjusts to different colored light each time and ascertains how much of a color’s light is reflected by an object, thus generating a complete spectral fingerprint of the object. The development engineers use a mathematical model to extract just about any information on an object, e.g. its constituents, from its spectral fingerprint. “Since hyperspectral cameras aren’t integrated in smartphones, we simply reversed this principle,” explains Seiffert. “The camera gives us a broadband three-channel sensor, that is, one that scans every wavelength and illuminates an object with different colored light.” This means that, instead of the camera measuring luminous intensity in different colors, the display successively illuminates the object with a series of different colors for fractions of a second. Thus, if the display casts only red light on the object, the object can only reflect red light – and the camera can only measure red light. Intelligent analysis algorithms enable the app to compensate a smartphone’s limited computing performance as well as the limited performance of the camera and display.

The first laboratory version of the app is finished. The engineers are developing a variety of initial applications before it can be released to private users, though. The system first has to be taught with reference scans how to analyze whether apples contain pesticides. Seiffert is hoping that the “HawkSpex® mobile” app might be launched on the market around the end of 2017.

Reference scans are not always needed, though. Some problems only require measuring different distributions of substances or materials rather than specifying individual constituents. Purchasing a car is one example: In this case, the app compares paint to determine whether it is exactly the same color everywhere or has been touched up.

Users will add applications – much like Wikipedia

“There are so many conceivable uses that the market will surely overrun us,” says Seiffert with certainty. That is why the development engineers are relying on an approach modeled after the online encyclopedia Wikipedia. “Once the app is launched on the market by the end of this year, active users will be able to contribute to the whole big thing and create new applications, for instance, that test pesticide exposure of heads of lettuce, by teaching the system such problems,” says Seiffert. This means they would use the app to scan different types of treated and untreated heads of lettuce and send the data to the Fraunhofer IFF. The development engineers would verify the measurements and release the app to all users.

The app has extremely interesting commercial potential, too, and can be used to develop sectors that wouldn’t really benefit from high precision scanners. Examples include quality control of foods, the effectiveness of cosmetic products or even agriculture: Farmers, for instance, could easily obtain information on whether their crops are sufficiently supplied with nutrients or fertilizer is needed.

Learn more: App reveals constituents