Breaking Down Big Data
Big data is a big deal, but what is it? Inbound Logistics asks four industry experts to file their report on big data and its benefits.
More to the Story:
Your sticky note and spreadsheet filing system has outlived its usefulness. But with so much data coming from your yard, warehouse, transportation management systems, logistics partners, suppliers, and sales team, what in the world do you do with it? All of this data combined is known as big data, and if you don't know how to process it all, it can give you a real headache.
Every shipper knows that big data is a big deal. It's not quite a household term, but it's certainly a phrase used in every office. But what does it mean? How does it apply to your operation? How can you use it to make intelligent, well-informed decisions? What kind of software do you need to curate your data?
It all seems intimidating, and without the proper guidance, it's hard to know where to begin. With so many questions on the table, Inbound Logistics gathered a few "Data Miners" to talk about trends and challenges in the business intelligence world.
Everyone talks about big data, but what is it? How do you define it as it relates to supply chain and logistics management?
FRODE HUSE GJENDEM: Big data refers to the increasing volumes of data from existing, new, and emerging sources—smartphones, sensors, social media, and the Internet of Things—and the technologies that can analyze data to gain insights that can help a business make a decision about an issue or opportunity. When applying big data analytics to supply chain or logistics management, the data insights can offer companies a strategic advantage. For example, companies can make data-driven decisions on how to improve customer service and demand fulfillment, or establish quicker and more effective reaction times to supply chain issues.
JOEY BENADRETTI: Big data, as the name suggests, is large sets of related data that may be scrutinized to reveal trends, patterns, and other relationships. While most enterprise applications can manage large data sets, the key with big data is not just the amount of data, but also its complexity and the difficulties of processing it utilizing traditional enterprise applications. In some cases, it could be comparatively smaller data sets with huge complexities.
Big data is produced from multiple sources, yet related in every aspect of its intended utilization. Consider the number of data points and origins connected to large and complex supply chains. Now, consider being able to bring all that together, and having it related to a few entities in the enterprise with the ability to see relationships. Also consider the importance of relating the data to a single business entity that helps with decision-making.
Data in the supply chain must be controlled to make it meaningful. This means that with the high throughput and complexities of absorbing all the data, the relevancy of what gets introduced and analyzed is critical to the true meaning of big data.
HAROLD B. FRIEDMAN: Big data is a popular term used to describe the exponential growth and availability of data, both structured and unstructured. Taking a practical approach, the business question you need to answer is: how do I create tangible outcomes based on practical opportunities using big data? Data needs to be condensed—presenting only what is relevant. For example, executives may want summary data across the company's product lines, or managers could want more detail, but only for the areas that they oversee.
Chief data officers or IT professionals are challenged with not only providing the infrastructure, but also working with others to provide meaning to big data.
ALEC CAMPBELL: Considering its position as the nexus of the supply and demand planning process, of all corporate functions, the supply chain likely collects the greatest amount of raw data. In the most elemental sense, big data, as we define it, forms the building blocks of the supply chain in transactions, movements, product details, and customer and supplier information. The difference between yesterday and today is not that there is more of it, it's the relative ease and speed with which the supply chain can create business insight with its data. In this way, big data isn't new to the supply chain, but the ability to quickly and cheaply leverage it is.
Where is big data located? Where can shippers mine it from?
GJENDEM: Data is everywhere. Through the rise of social media, smartphones, and the Internet of Things, new data types and sources are appearing every day, providing companies with the ability to discover new insights on how to best achieve their business goals. For shippers, structured data can be found in many places including enterprise resource planning (ERP), transportation management systems, and planning and procurement systems. Unstructured data can also be found in data sources surrounding the company, including financial data, weather data, or GPS data from trucks.
BENADRETTI: Big data originates from multiple entry points and is traditionally housed in multiple locations—servers, databases, or tables. The relationships are complex, and the complexity is what often drives the inability of organizations to understand it.
FRIEDMAN: Big data can be found in many data streams—both inside and outside the organization. Below is an example of a big data information stream. Each of these 10 actions leaves a data trail that is compiled to measure performance and create process efficiencies.
- An order is received from a client or a purchase order is sent to a vendor.
- The Warehouse Management System (WMS) notifies the warehouse of the product that needs to be shipped.
- An electronic load tender is broadcast to multiple carriers.
- A carrier is selected based on price, time performance, loss and damage experience, and billing accuracy. This information is gathered from prior shipments.
- When the carrier picks up the load, its terminal is notified of the pickup, so the product can be tracked.
- The WMS makes the appropriate adjustment to inventory.
- An electronic bill of lading is prepared and accepted by the carrier.
- An advanced shipping notice is sent to the receiver.
- A record of the shipment is passed to the payment process to serve as an authorization for shipment to be paid, and to accrue the expense until the freight bill is received.
- The carrier submits its freight bill via electronic data interchange, and it is audited, matched to the shipment authorization record, and staged for payment.
CAMPBELL: For the supply chain, internal sources of big data lie primarily in regional ERP systems, production systems, warehouse management systems, and transportation management systems. Unique to the supply chain, data can also be external with distributors/customers, suppliers, and the logistics providers between each party such as forwarders, contract warehousing, contract manufacturing, and customs brokers. Most shippers UTi serves tend to underestimate the business case needed to co-opt their key providers to share forecasting, movement, or other non-standard data. As a provider, we understand how and why this can be difficult. We proactively share with our shippers the data they might need, and help them communicate and collaborate externally.
What are the biggest challenges shippers face regarding managing big data?
GJENDEM: Shippers face two main challenges with big data—deciding on the desired insights-driven goal, and understanding how to extract the data signal from the noise that will support the desired outcome. To overcome these challenges, shippers should first establish a strong relationship between the business and IT. Effective collaboration is key for developing strategies that lead to data-driven business outcomes. Then, it's important for the shipper to gain an understanding of the data's potential to reach the goal. The shipper can gain this understanding by applying exploratory approaches to data to find correlations, patterns, or predictions that can be relevant for the business.
BENADRETTI: Often, shippers only know the immediate relationships; for example, where did it come from and where is it going? They may not know the true origin and end points. Because shippers do not have visibility of the entire supply chain, it is often difficult for them to understand the importance and sensitivity of products shipped.
Take the transportation of human organs. The shipper knows everything about the organ it is shipping and is able to help move the proverbial supply chain along. This is true for most specialized shippers, but not always for the shipper of consumer or other goods. So consider shippers that move bulk products across the ocean or the country. Without knowing the data points involved in the supply chain, they may not be able to appreciate the importance a customer or vendor is placing on the shipping process.
When shippers have better insight into the entire supply chain when shipping goods for multiple customers in the same containers or shipment manifests, they are better situated to deal with the difficulties of contamination, shelf life, and protecting corporate trademark secrets. They also gain the ability to make strategic and operational decisions that affect the supply chain.
Another impact is understanding all the touchpoints and higher costs that can emanate from areas such as insurance, security, and the utilization of faster ships to avoid sea pirates in certain shipping lanes.
FRIEDMAN: First, shippers must determine the type of information they need. Second, they need IT support to access this information. Third, they must commit resources to capitalize on the data captured.
CAMPBELL: Shippers typically assume they don't have the data, or don't have time to do the work, and both those positions are slightly off-base. The biggest challenges are almost always either confidence in the quality of data (this is different than not having data), or having the proper people resources on hand to manage or extract data for analysis by an outside provider (this is different than not having time). The quality issue is often related to legacy or mismatched global systems, and the scale of data cleanup needed. The people resources issue typically arises because few supply chains have adequate dedicated IT staff who know what data is available—and can be made available to a consultant or other provider. Both issues are reasons to ask a third-party logistics (3PL) provider to help innovate and collaborate.
What are some specific ways big data can impact supply chain—and ultimately, enterprise—performance?
GJENDEM: Building a modern data supply chain—enabling all data to flow smoothly and quickly through an enterprise—helps to cultivate data into a company asset that can positively impact the supply chain and the overall business in various ways.
For instance, shippers would be able to more effectively analyze data from different sources, such as contextual sensor data on shipped storage containers, and weather data to understand the conditions goods are being transported in. These data insights can help a company better understand whether a shipment will be on time and arrive with the right quality. This data-driven improvement in supply chain efficiency could increase revenues or improve customer service.
BENADRETTI: Big data impacts more than the technology or application used to deliver it. Having big data without the means to understand it removes a lot of decision-making from the supply chain. Obvious factors, such as delivery methods and interface options, are affected by the size and complexity of the available data, the ability and interpretation of which could eliminate delayed actions, and ultimately poor service or decision-making.
FRIEDMAN: Having better information within a warehouse impacts inventory because turnover rates can be better managed, popular items can be identified, out-of-stock items can be reduced, and companies gain improved visibility to who is buying what. As a result, they can reduce cost and improve the bottom line.
Big data also impacts transportation because shipping smarter allows companies to optimize selected service levels with performance required. They can also consolidate shipments, build round trips, and optimize carrier selection based on cost and performance. The result is reduced transportation expense that falls to the bottom line.
Also, having access to information that allows companies to have the right product available, and deliver complete orders on time, improves customer service and builds customer loyalty, which will increase sales.
CAMPBELL: We are using big data in two specific ways. One way is to identify and quantify supply chain risk. UTi worked with an automotive parts supplier to perform an expedited freight risk analysis using the supplier's shipment data, rates, and event likelihoods. The project resulted in the shipper re-ordering the procurement ranking of suppliers based on a new logistics risk factor. The key was the trust to share data.
The second way is network optimization. The network modeling shippers can perform today when the proper data is sourced is both advanced and comparatively low cost. One tool we use is LLamasoft Supply Chain Guru, which can generate rapid insight into current and future supply chain performance at the SKU and customer level. We can work with companies to model precise costs, service levels, and working capital needs for a two versus four distribution center model, or for switching to ocean freight for slow-moving products.
What types of data can create real value for shippers? In what area should shippers look to get the quickest ROI from analyzing big data?
GJENDEM: To get the most value from data, companies should establish a clear view of what will be a competitive differentiator for them, and an understanding of how their industry is evolving. Then they need to develop an actionable road map based on those insights. It is also important for businesses to pursue a single big data pilot or proof of concept first, then let the results cascade into a larger enterprise-wide strategy from there.
Specifically for shippers, big data can deliver value by helping them reduce shipping costs and increase service levels. To achieve these results, shippers should seek to understand the primary and secondary drivers of their costs and service level issues, then build a data model with linked data sources that can provide insights on how external and internal events are affecting their costs or issues. For example, shippers can use raw material or commodity price predictions to understand future fuel prices. These insights can support a fuel purchasing strategy and cost management. Additionally, boat operators can analyze weather and wave data to optimize shipping routes and fuel consumptions, and traffic data can provide dynamic input to effectively route delivery trucks.
BENADRETTI: Understanding costs provides the biggest value to any organization, and identifies where ROI might lie. Cost of product is not just the cost of making the product, but also the cost of moving the goods from Point A to Point B. Now, consider what delays might cost an organization in spoiled product, lost revenue due to poor customer service, and costs and tax implications related to shipping less product more often. Understanding all the key data points in the supply chain allows shippers to learn all the layers and elements of their cost structures, and helps them make smart decisions. In turn, shippers can increase revenues by making sure the right amount of product is shipped from the right locations using the right routes to deliver the right service on time, all the time.
FRIEDMAN: Big data ROI depends on the shipper's current access to information and if it is kept in separate silos or harmonized to maximize value. You do not have to spend all your time on the technical side of big data. You can broaden your horizons and concentrate more effort on discovering the business questions you need to answer. This will allow you to focus on practical opportunities with tangible outcomes. The best ROI can be found in inventory and transportation.
CAMPBELL: Inventory optimization is widely known to have a steep ROI and rapid payback, but the challenge is finding the data and being able to operate the system. UTi has an inventory optimization desk that collects millions of data points each day related to specific clients' sales/POS, transportation, warehouse SKUs on hand, new supplier orders, and supplier delivery. This center of excellence performs optimal replenishment planning for some innovative companies. This is an extremely ripe area for shippers, especially those who also warehouse, to collaborate for inventory optimization purposes.
How can shippers tap into big data for decision support? How can they extract it for strategic improvements?
GJENDEM: Big data and analytics should be shaped in a way that integrates it into the shipper's planning and operational processes. Once shippers perform analytics and achieve insights, they can make the resulting data-driven decisions in a few ways -- automated for projects such as route optimization, or manually for matters such as fuel contracting. For decisions having longer-term impact, shippers can look at developing proof of concept simulations that explore different internal and external variables to gauge where optimal impact can be realized.
BENADRETTI: Understanding all the data points in the supply chain can be transformed into information that impacts the decision-making process. For example, data can impact obscure areas such as how weather affects the transportation of goods by ships, and the degrees of water damage and lost cargo affected by the choice of transportation corridors. That allows decision-making on more than just where to manufacture a product or what raw material must be used, but also how to get the best quality product to the customer on time at a reasonable cost. It limits the ability to make strategic and operational decisions that affect the supply chain.
FRIEDMAN: Shippers need IT professionals on their team. Trying to capture key bits of information from fast-moving flows of big data is not easy. An organization must set priorities, identify opportunities, and determine the value proposition. Start small and look for little wins before making big investments in massive projects. This will require a change in culture—from just getting the product out the door to finding the optimum way to meet the needs of customers.
CAMPBELL: Shippers overestimate data availability issues, and sometimes lack the resources to execute on a big data project. So the answer to how shippers can tap into data and extract improvements is to ask for help. More shippers should invest time in understanding supplier capabilities to help extract and advise on big data projects. The advantage is a 3PL has likely done a similar project before, and can share cross-industry insight and best practices.
Shippers have always had concerns about the security of cloud applications, especially where proprietary information is involved. Are these concerns still real? What should technology buyers consider before they adopt hosted, cloud-based solutions?
BENADRETTI: Data security is always a concern, and as recent events have shown, it can be crippling to any organization that cannot secure its data. Company secrets are just as important.
By ensuring that the proper data is seen by the proper users, accidental sharing of this information can be reduced. Having a central access point for big data is only the first step in data security. With such an access point, you can provide a dual layer of protection and the ability to take corrective action if there is a data break.
CAMPBELL: The concerns are still very real from a risk mitigation perspective, particularly for consumer-facing businesses. Unfortunately, there is no magic bullet technology as cloud and on-premise systems both have similar security vulnerabilities. Solving this issue is a matter of sitting down with an objective adviser—not the software company selling a product—and studying the business need, risks, and possible solutions.