What do we do with all this data? It seems to be coming from everywhere and there’s more of it than ever before. Each day we’re making thousands of terabytes of new data. If only integrating it all was as easy as pushing a button! But it isn’t.
Let’s take a look at where this new data comes from and some of the problems it causes.
The Driving Force of Data
Organizations continue to collect more and more data. Whether it’s from internal systems such as core operating systems, users, external digital marketing sources, or ERP – there is more data now than ever. As executive-level initiatives create more data collection points, data silos need to be built to hold said data. The problem arises when the executives still want quick answers from all of this data regarding their business performance, which means data has to reside in the mission critical systems for quick access.
The Reality
What executives don’t always understand is the reality – that this data is from heterogeneous sources and it’s located in different places. From core operational sources, to ERP, to supply chain, to social media and even places as simple as Excel, Access and SQL server – it’s sometimes a fragmented mess!
Some organizations will still perform a few or all of these archaic and so-called data integration methods.
- Copying production databases or large database subsets is cumbersome and maintenance intensive.
- FTP is extremely maintenance intensive and antiquated. I.T. personnel, in job security mode, sometimes manually move data on a daily basis which is impractical in the 21st century and not to mention expensive.
- I.T. personnel creating scripts using rudimentary batch scripting languages at the operating system level is not a feasible solution either.
- Some organizations still maintain redundant sources of data essentially eroding any single source of truth.
With all these fragmented systems, businesses eventually figure out how to survive via survival systems and begin extracting data into more data sources such as Excel spreadsheets just so they can achieve somewhat of a consolidated view for identifying trends in business performance. All that effort is rarely a repeatable process for the next business cycle. If the organization doesn’t have the computer-savvy staff that can download and extract data, then pull it into Excel or Access, they have no other option but to live with partial chunks of data that cannot be related or integrated.
Fortunately, there are options available to help with these types of situations. If you’d like to talk more about what’s available, feel free to contact us for more information, and to answer any questions you have.
View Part 3 – Data Integration
Leave A Comment