Tracking International Development in the 21st Century Using Big Data

The office has the look and feel of an Internet start-up.

The workers are young, the dress is casual and the computer of choice is an Apple notebook. They inhabit a single open room. The walls have whiteboards for scribbling ideas when inspiration strikes.

But the office in Manhattan is not dedicated to the latest app. It is the base camp of the United Nations Global Pulse team — a tiny unit inside an institution known for its sprawling bureaucracy, not its entrepreneurial hustle. Still, the focus is on harnessing technology in new ways — using data from social networks, blogs, cellphones and online commerce to transform economic development and humanitarian aid in poorer nations.

“We work hard, play hard and tend to stay well-caffeinated,” said Robert Kirkpatrick, who leads the group. “This is an exercise in entrepreneurship.”

The efforts by Global Pulse and a growing collection of scientists at universities, companies and nonprofit groups have been given the label “Big Data for development.” It is a field of great opportunity and challenge. The goal, the scientists involved agree, is to bring real-time monitoring and prediction to development and aid programs. Projects and policies, they say, can move faster, adapt to changing circumstances and be more effective, helping to lift more communities out of poverty and even save lives.

Research by Global Pulse and other groups, for example, has found that analyzing Twitter messages can give an early warning of a spike in unemployment, price rises and disease. Such “digital smoke signals of distress,” Mr. Kirkpatrick said, usually come months before official statistics — and in many developing countries today, there are no reliable statistics.

Finding the signals requires data, though, and much of the most valuable data is held by private companies, especially mobile phone operators, whose networks carry text messages, digital-cash transactions and location data. So persuading telecommunications operators, and the governments that regulate and sometimes own them, to release some of the data is a top task for the group. To analyze the data, the groups apply tools now most widely used for pinpointing customers with online advertising.

“We’re trying to track unemployment and disease as if it were a brand,” Mr. Kirkpatrick said.

Read more . . .