Oracle Data Integrator combined with Hadoop, can be used to design the integration flow to process huge data from non-relational data sources.

Apache Hadoop is designed to handle and process data that is typically from data sources that are non-relational and data volumes that are beyond what is handled by relational databases.

You can use Oracle Data Integrator to design the 'what' of an integration flow and assign knowledge modules to define the 'how' of the flow in an extensible range of mechanisms. The 'how' is whether it is Oracle, Teradata, Hive, Spark, Pig, etc.

Employing familiar and easy-to-use tools and preconfigured knowledge modules (KMs), Oracle Data Integrator lets you to do the following:

• Reverse-engineer non-relational and relational data stores like Hive, HBase, and Cassandra.

  For more information, see Creating ODI Models and Data Stores to represent Hive, HBase and Cassandra Tables, and HDFS Files.

• Load data into Hadoop directly from Files or SQL databases.

  For more information, see Integrating Hadoop Data.

• Validate and transform data within Hadoop with the ability to make the data available in various forms such as Hive, HBase, or HDFS.

  For more information, see Validating and Transforming Data Within Hive.

• Load the processed data from Hadoop into Oracle database, SQL database, or Files.

  For more information, see Integrating Hadoop Data.

• Execute integration projects as Oozie workflows on Hadoop.

  For more information, see Executing Oozie Workflows with Oracle Data Integrator.

• Audit Oozie workflow execution logs from within Oracle Data Integrator.

  For more information, see Auditing Hadoop Logs.

• Generate code in different languages for Hadoop, such as HiveQL, Pig Latin, or Spark Python.

  For more information, see Generating Code in Different Languages

Big Data Knowledge Modules Matrix depicts the Big Data Loading and Integration KMs that are provided by Oracle Data Integrator.