Integrating Multiple Data Sources in a TMS
Q: What are some of the data management challenges to expect in a transportation management system (TMS)?
A:The most important thing is to decide what data you want/need, which trading partner has it, and how you will obtain it in a format that will be required for standardization of reporting and data mining.
Typically, for a TMS to obtain pertinent data elements, it requires integrating with numerous trading partners including carriers, PO management systems, customers’ ERP systems, WMS, visibility platforms, loadboards, accounting systems, insurance providers, other TMS/brokers, and more. Data from these sources can be delivered in a variety of methods and formats.
The development of a database schema and an entity relationship model to define and enumerate logical relationships between the data entities is needed. Normalizing the data to the lowest common denominator and working your way up will allow you to effectively integrate disparate data sources.
Q: How do you address the issue of dealing with different data sources?
A: Data integration in the purest sense is about carefully and methodically blending data from different sources, making it more comprehensive and therefore more useful and valuable than it was before.
To be successful, you need to be flexible in the method and formats of the delivery and receipt of data. Exchanging information from only one method or format limits your trading partners and can limit the amount of data available in the data warehouse.
Your data sharing technology needs to consider:
- The types of data and the size of the data sets that are shared
- The data processing services and tools that are required to maintain or use the data
- The technical capabilities and skills of data users and data providers
- The types and levels of data and access security that are required
- The software and hardware required by network users
- The level of effort and cost to develop and operate the exchange network
The answers to questions such as these address the overall feasibility of the data exchange network from a cost-effectiveness and likelihood of success perspective.
Q: What advice do you have for managing the integration?
A: Be flexible. Don’t be too rigid with your methods of communication or formats. Play to your trading partners’ strengths by utilizing the method they use that best meets your requirements.
For instance, the current trend in sharing carrier data is using APIs; however it is still important to be able to utilize other traditional methods such as EDI and flat file exchanges for quoting, tendering, tracking, and invoice data since some carriers still utilize EDI vs API. Without EDI or some other form of data exchange, you would be excluding data from those carriers, leaving an unacceptable void of data. As methods and formats of data exchange continue to evolve, a strategy that can accommodate a variety of means to manage the multitude of sources is prudent.