Audami ​data consultants
  • About
  • Contact
  • Process Mining
  • Blog

Turning small data into big data and vice versa

10/19/2015

1 Comment

 

Chess computers these days are much smarter than humans. Why is this true and are computers also smarter when it comes to management decisions?
If we look at a position on a chess board, we can analyse the data it includes. Actually it is a case of small data: we see 64 fields and pieces that occupy minimal 2 and maximal 32 fields.
The fields can be black or white, however this is irrelevant for playing the game; it is merely a nice visual effect that help humans to see the position.
The pieces can belong one of 2 armies: the white or the black side.
The pieces can be king, queen, rook, bishop, knight, or pawn, so we have limited choice of 6.
So if we know which pieces are on which of the 64 fields, we know all we need to make a decision on our next move. Not what you call big data…
However, we humans are experts in making big data from small data. First we give a value to the pieces: A pawn is 1, a bishop and a knight is 3, a rook 5, a queen 9 and the king has no value or infinite value, depending how you look at it, because without a king you are lost.
Then we argue that 2 bishops have more value in positions with less pieces, and that there are certain chains of pawns that are favourable, etc. Suddenly we (as we get more experienced) are starting to “see” all kinds of patterns, which of course can be captured in data. And, before you know it, there is big data.
Too play chess, you have to predict what the next moves will be, and to evaluate the position that will arise after 1, 2, 4, 8 etc. moves. Unfortunately, humans are not really good in capturing all these potential positions, and are already happy if they can “see” the position after one or two moves. Of course for a computer it doesn’t mind if the position actually is present on the chess board, or one of the potential chess positions.
To overcome their blindness, humans add all this data about structures, patterns etc. to the current position and then decide on a move. A strategy that can work well if there are no tactical pitfalls. Some grandmasters were experts in avoiding positions with pitfalls, other grandmasters, especially of the current generation, they like to try to outsmart their opponents by looking more moves ahead.
Chess computers nowadays do not mind too much about patterns and structures, but they are able to see 25 moves ahead. This way they can do much better than the strongest grandmaster.
What can we learn of this example, with respect to big data and management decisions?
Well human managers are likely to base their decisions on a lot of data, based on their experience they see all kinds of patterns and structures, however they are poorly equipped to look a few moves ahead, or: able to understand what happens in the market in the coming 1, 3, 5 years. So what we are doing is gathering all these data and let the computer sort it out.
However, before we do that, it would be wise to turn the big data into small data, and then let the computer predict the next moves of our customers, our competitors, our governments etc.
Of course the business “game” is not as simple as chess, and it is much harder to distinguish the small data. But I believe that the most successful data scientists of today are the ones that are able to see the “small” data in your big data and then use this data to calculate which decision is the best.
 

 


1 Comment
Eline link
12/10/2015 01:19:56 pm

Nice story, though the English is a little less than perfect. I know nothing about data mining but I could perhaps be of assistance from a strictly linguistic point of view ;-)

Reply



Leave a Reply.

    Author

    René int Veld is a data scientist with 30+ years of experience.

    Archives

    October 2015

    Categories

    All

    RSS Feed

Powered by Create your own unique website with customizable templates.