This blog post was originally posted on the Informatica website on October 15th, 2014 here:
http://blogs.informatica.com/perspectives/2014/10/15/informatica-brings-enterprise-wide-governance-and-management-to-hadoop-with-cloudera-integrations/#fbid=GN4ScbY72Yd

0

Informatica Brings Enterprise-Wide Governance And Management To Hadoop With Cloudera Integrations

Posted on October 15, 2014 by Scott Hedrick

inShare77

Governance and management of Hadoop clusters in enterprise environments has been a challenge for organizations that want to put data pipelines into production use.  Without proper tools, organizations are forced to rely on their teams to thoroughly document data pipelines and hope that no one will show up to audit their Hadoop data infrastructure.  Without end-to-end management, apps running on the Hadoop cluster need to be installed and updated manually.

Informatica’s recent integration with Cloudera Navigator and Manager are major steps forward in enabling enterprises to confidently and efficiently deploy Hadoop and data management tools into production with their existing enterprise data infrastructure.

Over the past decade, Informatica has been developing the most sophisticated and flexible metadata management in the industry. Thousands of enterprise customers rely on Informatica to help them govern their data and comply with industry regulations. By integrating with Cloudera Navigator, Informatica has extended it’s proven metadata management tools with the ability to view end-to-end data lineage of data flows across entire data pipelines that flow from source systems into and processed on Cloudera Hadoop clusters and into target applications.

Informatica’s metadata management capabilities combined with Cloudera Navigator is the most advanced solution today for tracking metadata within Hadoop and across other enterprise systems.  With the ability to pull data in from Navigator, Informatica can visually trace full data lineage starting from the source systems outside of Hadoop, through the data integration, parsing and cleansing that is executed in the Hadoop cluster and on to target analytical systems outside of Hadoop. When the processing on Hadoop is done with Informatica Big Data Edition, all the details of each transformation, data mappings and associated metadata is brought together to provide governance of the entire data pipeline. Each step can be drilled into for greater detail on the transformations in the data wrangling process and the resulting data relationships in the curated end results.

For organizations that have been looking for the same level of governance and management from their Hadoop enterprise data hub that they have come to expect from their tradition infrastructure, this is the solution. Data scientists and analysts can be more certain of the content and veracity of their data by getting a detailed view into where their data has come from and how it has been combined, parsed, cleansed and prepped for analytics projects. IT organizations needing to comply with industry regulations regarding data can be confident that everything is in place to trace the complete data history for audit purposes. The new levels of data visibility and governance this integration provides with the combination of Informatica and Cloudera dramatically boosts the confidence organizations can have about putting Hadoop into production.

In order to make it easier for customers to manage deployments of Informatica Big Data Edition on Cloudera Hadoop, Informatica has integrated with Cloudera Manager. Using Cloudera Manager, Informatica Big Data Edition software components can be installed on each node of the Cloudera Hadoop cluster for distributed processing leveraging the full power of the cluster.  Big Data Edition patch levels can be monitored and the software updated on the Cloudera cluster via the Cloudera Manager interface.  This makes it easier to manage Informatica Big Data Edition as an integral part of a Cloudera Enterprise Data Hub.

Come and see full Hadoop data pipeline lineage in action!  Informatica will be showing demos of the new integrations with Cloudera Navigator and Manager in the Informatica booth #352 at Strata + Hadoop World in New York City.  Contact Informatica if you would like to schedule a time to get more information about how complete data lineage and manageability can add value to your organization.

For more information, here is a link to Informatica’s press release announcing these new capabilities.