Data Exhaust, Ladders, and Search

I spent Thursday at the Palo Alto Research Center (PARC) in California, where I gave a talk. My subject? Data exhaust — the unintended information we throw off in our daily activities. Much like the information generation by trading equities, currencies, and bonds, the broader world is full of information that we generate unintentionally by going about our lives. With that in mind, my talk ranged across Twitter, California highway debris and the Google’s search algorithm troubles.

I won’t reiterate the whole talk here, but I’ll put something forward as worth pondering.: What are the consequences of an instrumented planet? In the financial markets we have long been used to the idea that data flows constantly, some of it spurious, some of it meaningful. But we are heading down a path toward a planet where pretty much everything throws off data, not all of it intentional. Taking that further, statistically speaking, the more data we see from more instruments, whether intentional or unintentional, the more likely some of it is extreme, even if only by chance. What are the consequences?