I’ve harped on the use of big data in the supply chain a few times already. In fact, historically one of the issues with the supply chain in general and EDI specifically is the amount of data generated by the plethora of transactions moving between trading partners. And as the demands for visibility increase so does the number and complexity of the documents.
A couple years ago I looked into some issues a large retail supplier had with some of their processing. One of my recommendations was to increase the amount of historical data they were storing so that they would be able to look backwards in time to the actual documents that were being questioned by their customers. At the time they were only maintaining a 90 day history. That meant any disputes regarding transactions older than 3 months became difficult or impossible to respond to in any meaningful way.
That recommendation was made during a time when storage space was becoming significantly less expensive, but before the concept of big data had really taken hold as a standard – or at least a reasonable option for data retention.
I came across a report from McKinsey & Company that discussed the concept of big data and a study they had performed. The report dates back to 2011 and covers 5 domains. One sector they studied was “retail in the United States.” Even at that time (before many new tools and storage techniques, not to mention cost reductions) the report indicated, “Big data can generate value in each [sector]. For example, a retailer using big data to the full could increase its operating margin by more than 60 percent.”