Augment Your Data

Extract and load data from your custom transactions and make it readily available in tables populated in the autonomous data warehouse.

You can use the system provided or customer provided source tables that are the custom transaction objects that you created in NetSuite. The system provided tables are pre-validated by Oracle NetSuite Analytics Warehouse. The customer provided tables are other source tables that are available for extraction but aren’t validated by Oracle NetSuite Analytics Warehouse. As a user with the functional administrator or system administrator application role, you can allow usage of a particular table that isn’t pre-validated by Oracle NetSuite Analytics Warehouse. However, Oracle can't ensure the success of processing such custom tables or any performance impacts, such as delays in the daily refreshing of data.

If you enable the SME Options for Data Augmentation under the Generally Available Features tab on the Enable Features page, then you can augment your reports with datasets created by extending an existing entity or group of facts, by adding a new dimension in the target instance, and by adding a new fact in the target instance. When you run these data augmentation pipeline jobs, they publish these datasets to the semantic model. However, this isn’t the recommended practice. The recommended method is not to enable the SME Options for Data Augmentation feature and use the default Dataset augmentation type to bring varied data into the warehouse. When you run the Dataset data augmentation pipeline job, it doesn’t publish anything to the semantic model. You can then use the semantic model extensions to create your own semantic model. This method supports complex semantic modelling to meet your business requirements. Use the Data augmentation capability to bring data into the warehouse and then use the Semantic Model Extensibility capability to create the joins and expose that data to the subject areas that you want. This enables flexibility and better performance of both the capabilities. Additionally, this method allows better lifecycle management. For example, if you need to make any adjustments to the semantic model, then you can make the changes directly in the semantic model. You don’t need to adjust the data augmentation that brought the data into the warehouse.

The Dataset augmentation type isn’t associated with any other augmentations. Based on the incremental schedule, the data in this dataset gets refreshed during scheduled pipeline refresh. But unlike other augmentations, this augmentation isn’t linked to other augmentations, and you can’t change the attributes as dimension or measure. This dataset isn’t associated with any subject area, since it is simply copying the dataset from source and creating a warehouse table. You can perform semantic model extension after the table is created. To use this dataset to build the joins or incorporate an object from the dataset into your semantic model, you must run an incremental load prior to using it because the incremental load populates the dataset.

You see the data augmentation pipeline jobs on the Data Augmentation page with one of these statuses:
  • Activation in Progress - You can’t edit, delete, or schedule a data augmentation pipeline job while activation is in progress.
  • Activation Completed - You can edit the data augmentation to add or delete VO attributes and save the changes. You can’t modify the schedule in this status.
  • Activation Scheduled - You can edit the data augmentation to add VO attributes, save the changes while retaining the existing schedule, reschedule the execution date and time, or execute the plan immediately.

Note:

You can change the names of the columns that you’ve added from the various data sources in your data augmentation. Later if you delete a data augmentation, then you must wait for the daily incremental run to complete to see the change in the reports, cards, and decks.
  1. In Oracle NetSuite Analytics Warehouse Console, click Data Configuration under Application Administration.
  2. On the Data Configuration page, under Configurations, click Data Augmentation.
  3. On the Data Augmentation page, click Create.
    You see the data augmentation wizard with multiple steps. You can create the Dataset augmentation type by default. However, if you enable the SME Options for Data Augmentation under the Generally Available Features tab on the Enable Features page, then you can select the Create Dimension, Create Fact, and Extend Entity type of augmentations. To create these augmentation types, ensure to enable the SME Options for Data Augmentation feature. See Enable Generally Available Features
  4. In step 1 of the wizard, select an augmentation type. Each augmentation type requires you to complete certain tasks.
  5. For the type of augmentation, if you select Create Dimension to add a new dimension in the target instance, follow these instructions:
    1. Select Custom Record in Source Dataset Type.
    2. Select NetSuite Warehouse in Pillar.
    3. In Source Table Type, specify the source table type using either of the options and then click Next:
      • Select System Provided and then in Source Table, select a table to which you want to add the new dimension.
      • Select Customer Provided and then in Source Table, enter the name of the table to which you want to add the new dimension.
    4. Optionally, select the Versioned Dataset check box to enable full load of the source table data everytime and then click Next.
    5. In step 2 of the wizard, in the Available Attributes pane, select the check box for the attributes that you want in your new dimension, and then click Add Selected.

      You see the attributes that you had selected in the Target attributes pane and the recommended (defined) primary key. You can either accept this key or override it with your own primary key definition. You can mark any columns as incremental to ensure that you get new data specific to the columns.

    6. Optionally, select Advanced to reorganize the order of columns that are marked as primary keys. To reorder, select a primary key row and then drag to define a suitable join order.
    7. Click Next.
    8. In step 6 of the wizard, provide the following details and click Finish to save and schedule your data augmentation pipeline job:
      1. Name your augmentation pipeline job; for example, Sales Order.
      2. Enter a suffix for the target table name using underscore in place of spaces between words and don’t use special characters; for example, Sales_Order_D. The augmentation process automatically creates the target table name.
      3. Provide a description.
      4. Select the functional area and one or multiple subject areas in which you want to include this augmentation pipeline job.
      5. Specify the options to save the data augmentation pipeline job without executing it, or schedule the execution date and time, or execute it immediately.
  6. For the type of augmentation, if you select Create Fact to add a new fact table in the target instance, then follow these instructions:
    1. Select Custom Record in Source Dataset Type.
    2. Select NetSuite Warehouse in Pillar.
    3. In Source Table Type, specify the source table type using either of the options and then click Next:
      • Select System Provided and then in Source Table, select a table from where you want to add the new fact table.
      • Select Customer Provided and then in Source Table, enter the name of the table from where you want to add the new fact table.
    4. Optionally, select the Versioned Dataset check box to enable full load of the source table data everytime and then click Next.
    5. In step 2 of the wizard, in the Available Attributes pane, select the check box for the attributes that you want in your new fact table, and then click Add Selected.

      You see the attributes that you had selected in the Target attributes pane and the recommended (defined) primary key. You can either accept this key or override it with your own primary key definition. You can mark any columns as incremental to ensure that you get new data specific to the columns.

    6. Optionally, select Advanced to reorganize the order of columns that are marked as primary keys and specify one of your incremental keys to use when determining the initial extract date.
    7. Click Next.
    8. In step 6 of the wizard, provide the following details and click Finish to save and schedule your data augmentation pipeline job:
      1. Name your augmentation pipeline job; for example, Sales Order Line.
      2. Enter a suffix for the target table name using underscore in place of spaces between words and don’t use special characters; for example, Sales_Order_F. The augmentation process automatically creates the target table name.
      3. Provide a description.
      4. Select the functional area and one or multiple subject areas in which you want to include this augmentation pipeline job.
      5. Specify the options to save the data augmentation pipeline job without executing it, or schedule the execution date and time, or execute it immediately.
  7. For the type of augmentation, if you select Dataset to copy a dataset as is into a target warehouse table, then follow these instructions:
    1. Select Supplemental Data in Source Dataset Type.
    2. In Source Table Type, specify the source table type using either of the options:
      • Select System Provided and then in Source Table, select a table for the attributes.
      • Select Customer Provided and then in Source Table, enter the name of the table for the attributes.

      Note:

      You can select your customer extensions, item extensions, vendor extensions, partner extensions, and employee extensions as source tables if you want to add attributes from these entity extensions.
    3. In Source Table, select the applicable table whose attributes you want to add into the dataset and then click Next.
    4. Optionally, select the Versioned Dataset check box to enable full load of the source table data everytime and then click Next.
    5. In step 2 of the wizard, select the check box for the attributes from the source table to add to the target table, and then click Next.
    6. In step 3 of the wizard, select the settings for the selected columns, and then click Next.
    7. In step 6 of the wizard, provide the following details and click Finish to save and schedule your data augmentation pipeline job:
    8. In step 3 of the wizard, map the extension attributes to the warehouse entities by selecting the warehouse entity that is being extended. You see the primary column of the warehouse entity mapped with the attribute that you are extending.
    9. In step 6 of the wizard, provide the following details and click Finish to save and schedule your data augmentation pipeline job:
      1. Provide a name and description for your augmentation.
      2. Enter a suffix for the target table name using underscore in place of spaces between words and don’t use special characters; for example, Customer_Deposit_EXT. The augmentation process automatically creates the target table name.
      3. Specify the options to save the data augmentation pipeline job without executing it, or schedule the execution date and time, or execute it immediately.
  8. For the type of augmentation, if you select Extend Entity to add attributes to dimensions or measures to facts, then follow these instructions:
    1. Select Custom Attribute in Source Dataset Type.
    2. Select NetSuite Warehouse in Pillar.
    3. In Source Table Type, specify the source table type using either of the options and then click Next:
      • Select System Provided and then in Source Table, select a table for the attributes.
      • Select Customer Provided and then in Source Table, enter the name of the table for the attributes.

      Note:

      You can select your customer extensions, item extensions, vendor extensions, partner extensions, and employee extensions as source tables if you want to add attributes from these entity extensions.
    4. Optionally, select the Versioned Dataset check box to enable full load of the source table data everytime and then click Next.
    5. In step 2 of the wizard, in the Available Attributes pane, select the check box for the attributes from the source table that you want in your target table, and then click Add Selected. You see the attributes that you had selected in the Target attributes pane and the recommended (defined) primary key. You can either accept this key or override it with your own primary key definition.
    6. Select Advanced to reorganize the order of columns that are marked as primary keys and specify a date or timestamp data type column as one of your incremental keys to use when determining the initial extract date.
    7. Click Next.
    8. In step 3 of the wizard, map the extension attributes to the warehouse entities by selecting the warehouse entity that is being extended. You see the primary column of the warehouse entity mapped with the attribute that you are extending.
    9. In step 6 of the wizard, provide the following details and click Finish to save and schedule your data augmentation pipeline job:
      1. Name your augmentation pipeline job; for example, Customer Deposit.
      2. Enter a suffix for the target table name using underscore in place of spaces between words and don’t use special characters; for example, Customer_Deposit_EXT. The augmentation process automatically creates the target table name.
      3. Provide a description.
      4. Select the functional area and one or multiple subject areas in which you want to include this augmentation pipeline job.
      5. Specify the options to save the data augmentation pipeline job without executing it, or schedule the execution date and time, or execute it immediately.