Mining Meaning From Information

If the phrase "data mining" does not yet crop up in your daily conversations, it will soon. And if students graduating from your college or university are not familiar with it, your institution is doing something wrong.

Data mining is the use of computer algorithms to discover hidden patterns and unsuspected relationships among elements in a large data set. It arose at the intersection of artificial intelligence and statistics, and in many ways it is the technological answer to the problem of information overload: too many texts, too much data, not enough time to read and digest everything. Data-mining software locates the data we are interested in and presents them to us in understandable ways. It permits what people in the field call "knowledge discovery": finding meaning in what would otherwise be unmanageable amounts of information.

We have all benefited from, or suffered the consequences of, data- mining technology. Data-mining systems help financial institutions and businesses detect suspicious activity on credit cards, approve or deny credit or mortgage applications, and identify potential customers for a particular service or product. They even predict whether those customers would respond better to a mass mailing, a telephone solicitation, or another kind of communication.

Data mining can also help retailers single out particular customers with special offers based on prior purchases and buying patterns. The mantra of today's data-driven marketing firm is to send the right offer to the right customers at the right time through the right channel. Marketers' use of data mining explains why the offers we receive for new products or contract renewals, for instance, are so personalized, often vastly different from those received by our next-door neighbors. For example, as an incentive to renew...