You can’t get insights using big data techniques without the data. That much is clear. But where, exactly, is the data you need? Where does it reside, and how can you get access to it? The answers to those questions are not always directly obvious. But with the help of data catalogs, organizations are discovering that data doesn’t have to be so hard to find after all.
More than a decade into the “big data” era, we’re finally figuring out that Hadoop isn’t the answer to all of our data problems. Instead of centralizing data in giant HDFS clusters meant to serve the data needs of entire companies or departments, organizations are once again building one-off systems to handle specific data storage, processing, and analytic tasks.
To read the entire article, please click on https://www.datanami.com/2019/08/07/data-catalogs-seen-as-difference-makers-in-big-data/