I'm down with Big Data. I am.
Who isn't? Big Data is everywhere. Not only are we all down with Big Data, we all are Big Data, voluntarily or not providing an unfathomable amount of information to the big cloud in the sky, which tracks our needs, wants, habits, weaknesses, locations and destinations through smartphones that are sometimes too smart for our own good.
I know there is no stopping Big Data and most of me figures that's fine. But I do have a nagging worry. I worry that Big Data will mean the death of randomness, spontaneity and the joy of the unexpected.
Before I get to that, I should admit that until recently Big Data was little more than a buzz phrase to me. I knew it was important, omnipotent and omnipresent, but my little brain was no match for trying to understand Big Data. I mean, what exactly do you do with all that info?
Then I met Jim Meyerle, co-founder of a Big Data company called Evolv. Over a cup of coffee he explained that his San Francisco firm is using Big Data to help other companies figure out how to hire the right hourly workers for the job (and keep them on the job longer than your average hourly worker).
"What we do," he says cheerfully, "is we allow employers and employees to better communicate with each other."
And through surveying workers and tracking how they behave and mixing in some data from the federal government and expertise from workforce scientists, Evolv determines that things are true that have no business being true.
Like the fact that ex-cons make good employees. Really. Hire them before your competition does.
"When you look at productivity and attrition," Meyerle says, "convicts perform as well as anybody else and under certain circumstances, they actually perform better."
So, Big Data: a friend to ex-cons.
Of course companies unleash Big Data to make big decisions. Big Data is about big problems. Companies use it for sales leads. Police departments use it to thwart criminals. Scientists use it to predict earthquakes, hurricanes and global warming. Presidents use it to figure out who might be persuaded to vote for them and Nate Silver uses it to tell us who our next president is going to be.
But my worry stems more from how Big Data is increasingly telling us who we are and what we want. Google (GOOG) News tells us what news stories we want to read. Amazon tells us what books we want to buy. iTunes tells us what songs we like, whether we like it or not. Waze tells us the way we want to drive home. Facebook tells us who are friends are. Are we very far from the day when Big Data tells us who we will marry, what we'll do when we grow up, where we will live and when we will die?
In a world of Big Data, what becomes of bumping into a long-lost friend at the coffee shop you're trying for the first time, or discovering a book on a shelf by an author you'd never heard of? What becomes of stepping away from the computer, starting on a walk around the block and ending up strolling through a park at the edge of your neighborhood?
What becomes of our moments that literally don't compute?
"You could think of it as taking away spontaneity, but you could also think of it as sharpening the pencil," says Rick Smolan, the Marin County photographer who compiled "The Human Face of Big Data," a massive coffee table book that looks at more than 100 stories of Big Data in action.
I'd called Smolan for reassurance. See, he sides with the 53 percent of the tech experts surveyed by the Pew Internet & American Life Project who said the rise of Big Data would be a big positive for society. Think about it, he tells me. Life can be defined as "little moments of brilliance and an awful lot of wasted time." What if by deploying Big Data you could increase the chances of creating a moment of brilliance and reduce the amount of wasted time?
"Imagine if the whole human race has been looking through one eye for all of humanity," he says, "and all of a sudden scientists have figured out how we can open up a second eye."
There is so much more to see out there, Smolan's photographer mind figures, and Big Data is the way to see it.
I know he is right. Big Data will bring us to places we can't even imagine now. It will help cure cancer, unclog freeways, ease homelessness, hunger and poverty. It will up your odds of winning at poker and help you decide whether to carry an umbrella to work. It will scheme with you to save energy at home and one day drive you to work.
But hopefully, Big Data will also allow us to turn it off, just for a bit, so we can let our minds and feet wander into discoveries that we are guided to only by our muse.
Contact Mike Cassidy at firstname.lastname@example.org or 408-920-5536. Follow him at Twitter.com/mikecassidy.