Big Data. Big deal?

Everywhere you turn you find someone is writing about big data.  The cover of Forbes, the Wall Street Journal, Time, and many others are covering the topic.  But, what is big data, and more importantly, why should I care?

Let's start with the definition....  Big data is well.... big.  But the real definition lies in not only volume, but also variety and velocity.  Variety meaning that it comes from many sources.  Look at businesses for example.  For years they have been reporting and analyzing their internal financial data.  There is a lot of it, but it comes from just one place - their internal system.  Now imagine combining that data with external data like Twitter feeds, weather information, population demographics, etc.  The insights you draw from the data may be different when you make it "big".


What about this velocity thing?  Are we about to have a physics discussion?  No. there won't be discussion of the V = D/T equation here.  For data, velocity is about the rate at which it changes.  The change in the data allows for better analysis over time because the sample size is greater.  Velocity plays a significant role in making the data "big"

So why should we care?  Is this about a bunch of computer geeks playing with 1's and 0's in a new way to create more job security?  No, not this time.  This big data stuff is real.  By using data that changes often, comes from disparate sources, and has enough samples to draw conclusions you can derive insights that were not easily made without big data. 

Just look at our last presidential election.  The Obama team spent millions of dollars to use big data to help win the election.  They created a team of data scientists, tucked them away in a windowless office in Chicago and asked them to take information from various sources to help predict and manage donor and voter behaviors.  They measured and tracked everything.  From the effectiveness of who an email piece came from to the tendencies of online donors.  When campaigners went door-to-door they knew in advance what type of information to leave behind.  They knew that emails from Michelle Obama were much more likely to raise money than from Joe Biden when sent to a specific demographic of prospective donors.  Not because Joe Biden is a putz, but because the numbers were right there in front of their eyes.  Television ad content, times, durations, etc were set as a result of the data, not some gut feel.  They were using the data to drive their decisions.

These data efforts helped the Obama campaign raise over $1 billion and ultimately know days before the election which swing states they were going to win.  They knew the outcome of the election well before the first ballot was cast because they knew the voters.  The data told them what the voters were going to do.

The sources for data are wide ranging.  The US government has a web site dedicated to data - data.gov.  It may not always meet the velocity requirements - you may have heard that they can be a little slow at times - but the variety and volume are there.  Access to information on Twitter, Facebook, and other social sites can provide great insights into trends and behaviors.  Digital marketplaces are popping up with people selling access to their data. 

 
With the digitization of our world, the volume, velocity, and variability of data will only grow.  Those that will succeed will be those who embrace the concept of big data and use it to drive value.  Simply put, big data can translate to big opportunities.

New Gadget - Ping Pong for ONE

I like playing ping pong.  I am not a great player but do enjoy the competition.  One of the biggest challenges is that you need a second person to play.  To make it truly enjoyable, that second person should be of equal or better skills than you.  In my house, this is easier said than done.

Growing up we had a table set up on top of our pool table.  You purists out there are scoffing already because the table is too high.  Get over it.  It's equally high at both ends.  Anyhow, we played a lot with the big prize being a game with dad.  He had developed some skills while in the Air Force and it was many years before we could actually beat him.  My brothers and I were pretty competitive and there were many heated contests.  One brother liked to play the junk game - spins, trick serves, etc.  He had to have the sandpaper racket and on occasion liked to fire it in your direction if things weren't going his way.  The ends of the table had the classic chips due to the frustrated racket slam.

When I went to college, we had a table in the dorm my sophomore year.  A group of us played quite a bit that year.  It was a good mix of players with pretty similar skills.  This made for great games.  After that, the opportunity to play took a hiatus for a while.  I was busy chasing other dreams.  After buying our first house we got a table.  I did manage to find a few people to play (my wife isn't too bad) but not as often as I would have liked.  That table was ultimately lost to basement flooding (water and fiber board don't play nice together) and was never replaced.

I had done some work for Disney at their animation studios in Anaheim.  Their techies were using our software to help in the creation of their animated movies.  Besides all the cool animation story boards on the walls, I made note of the two ping pong tables that were set up in a large common area.  Being an innovation and creativity center, they told me that this along with the video game machines were a great way to get the creative juices flowing.  They even had a ping pong league set up with the winner playing then company president Michael Eisner for the Disney championship.  Very cool. 

Shortly after that visit, we moved into a new office building.  One room had a non-functional fireplace and hardwood floors.  We didn't need it for offices and the 24 foot length was perfect for a table.  The game was back!  We played nearly every day.  It turned out to be a great meeting place and outlet for creativity.  Great games and great discussions.  The business was actually better as a result of its addition.  Most in the office played and the games were competitive sans one dude that could beat anyone playing right OR left handed.

That fun ended over a dozen years ago and brings us to today.  Our most recent house came with a pool table.  It is a beautiful table that has gotten very little use.  We were going to host a holiday party recently and my wife insisted (really) that we get rid of the pool table and have a ping pong table up and running for the party.  I was all for the idea but complained that it would get even less use than the pool table as nobody would want to play.  I got the "look" and headed out to find a table.  Rather than disassemble the pool table I chose to get a top for it.  This was easier than taking the pool table apart and gave options for down the road.  The problem however still existed - who to play?

Enter the Newgy Robo-Pong 2040 (www.newgy.com)!  This little gem was the answer to my long standing problem.  It also took care of finding a Xmas gift for someone (me) who apparently is very hard to shop for.  As soon as the table was set up I went online and ordered the Robo-Pong.  It arrived in just a few days and initial setup was less than 15 minutes (after that setup/takedown is less than a minute).  I applied power to the control unit and I was up and running.  The  wonderful design of this unit not only spits the balls at you but it comes with a catch net and ball feeding system that keeps those balls coming at you until your arm falls numb or you turn it off.  You can control the speed of the ball, the frequency of the serves, and the speed of the oscillation from the controller that sits on your side of the table.  The oscillation can be manually set from narrow to the full width of the table.  My unit came with 96 balls, plenty to keep you going for quite a while.  At higher speeds and oscillation, the balls do tend to fly around the room from all of the mis-hits but practice makes perfect.  This is one of those gadgets that works just as advertised.  If my sore shoulder is any indication, it can give you quite the workout.  Now if only I could convince the family to work on their game!