Configure Incremental Processing in a Data Flow
Configure incremental processing to load only new or updated records from a database. You can deploy incremental processing if your data is sourced from a database (using a database connection).
Incremental processing keeps your data up-to-date, providing better quality insights.
Before you start, create a connection to one of the supported databases, for example Oracle, Oracle Autonomous Data Warehouse, Apache Hive, Hortonworks Hive, or Map R Hive.
- Specify a new data indicator in the data source. See Specify a New Data Indicator for a Data Source.
- Configure a data flow for incremental processing. See Configure a Data Flow for Incremental Processing.
You can now schedule your data flow to load new data regularly.