Where is Frodo Baggins?

How can we ever hope to understand very large aggregates of anything? The first step, in any such venture, is to collect statistics. Once such collection is institutionalized—and data become available over some span of time—changes in the measured category may be detected by analysis. And any action taken to change or influence that aggregate can also be tracked—again with statistics.

Around here we fondly remember Daniel Patrick Moynihan, the activist Democratic senator—also a sociologist. In his engagement in the War on Poverty, he always urged the Bureau of the Census or the Bureau of Labor Statistics to set up new data collections. He knew full well that managing mass phenomena begins with numbers.

But collecting information is expensive. Suppose, however, that you’re already collecting it, but for another reason. Supposing that, in keeping records, a company captures hundreds, thousands, millions of various kinds of transactions—and saves these data for legal purposes. It was and still is done. Back in the ancient times these data were held on paper. In the early computer age on tape. It took a while before the secondary value of such data came to be recognized. I don’t remember when exactly—but my guess is that it was in the 1980s—data mining began to surface as a useful method of milking some value from these “givens.”  (Anciently I looked up data and discovered what the word means—just facts as recorded.)

The term used back then, mining, was quite appropriate in an age when old records were on cumbersome magnetic tape—and you had to read, rather slowly, many, many of them to get, say, ten years’ worth of history. After that analysis could begin—and new patterns would emerge.

Since the 1980s, massive technological changes have caused the transformation of ho-hum data mining into the new celebrity industry of Cybernetic Spying. Two new stories in today’s New York Times reminded me of that. One talks about NSA collecting images of faces from the electronic traffic; recognition software is in full development. The other is a story about a company called Palantir Technologies. The company is privately held but valued at around $9 billion; the story deals with the impatience of investors in the company because Palantir is unwilling as yet to go public. The company’s business? Cybernetic spying. The company’s name comes from Tolkien. A palantir is a magic stone that a very corrupt wise man, Saruman the White, uses to spy on the world. Very interesting choice of company name. The company, with Pay Pal figures active in its initiation, was initially funded by the CIA which, curiously—first time I’ve ever heard of it—has a “venture fund.”

Some fiction writers have quite prophetic powers—and Tolkien was one. We already have at least one palantir—and no doubt there are actually dozens. But where is Frodo Baggins? And never mind Gandalf the Grey…