Setting Processing Parameters for Section Sources

LATAM File Builder uses map/reduce scripts to extract data from the source and distribute it into input data files. Input data files store data in its original format, before processing.

The system also runs map/reduce scripts to process the content from input data files and apply the format defined in the file layout. This process generates summarized files.

To create the final output file, the SuiteApp places the content of the summarized file in the specified order.

Processing parameters define the size and length of the input data and summarized files created from the section source. Additionally, the parameters set the amount of files allowed per map/reduce script execution.

All parameters are optional and the system has internal default values set for each of them. SuiteApps that provide predefined file layouts may set custom values for the processing parameters.

Important:

You should not set or change parameters values unless instructed by NetSuite Customer Support.

To set processing parameters for sections sources:

  1. Go to Setup > Latam File Builder > File Layout.

  2. Click View next to the layout you want.

  3. Click the Sections subtab.

  4. Click the name of the layout section you want.

  5. Click the Sources subtab.

  6. Click the name of the section source you want to edit.

  7. Click the Processing Parameters subtab.

  8. If you want to set one ore more processing parameters for input data files, do the following:

    1. In the Number of Lines per Input Data File field, enter the maximum number of lines per input data file.

      Input data files store the data retrieved from the source before processing. When the number of lines surpasses the stipulated maximum, the system divides the content into several smaller input data files.

    2. In the Number of Lines from Input Data Files per Execution field, enter the number of lines from an input data file that a map/reduce script can process per execution.

      If the number of lines in the file surpasses the stipulated limit, the system processes the remaining lines in subsequent map/reduce executions.

    3. If you want to change the size that a folder must have to qualify for standalone processing, in the Minimum Folder Size for Standalone Processing field, enter the minimum folder size in KB.

      NetSuite creates folders to store the input data files from each section source. By default, to optimize map/reduce schedules and executions, the system processes folders in batches of up to 10 MB. For folders larger than 10 MB, it runs standalone executions.

    4. In the Number of Input Data Files per Execution field, enter the maximum amount of input data files allowed per map/reduce script execution in your account.

      A large number of input data files in a single map/reduce execution may consume a significant amount of the account's governance points. For more information, see Map/Reduce Governance.

  9. If you want to set one ore more processing parameters for summarized files, do the following:

    1. In the Maximum Size of Summarized Files field, enter the maximum size in KB of each summarized file.

      If the file size surpasses the stipulated maximum, the system divides it into several smaller summarized files.

    2. In the Number of Lines per Summarized File field, enter the maximum number of lines per summarized file.

      Summarized files store data after the processing of the input data files. When the number of lines surpasses the stipulated maximum, the system divides the content into several files.

    3. In the Number of Lines from Summarized Files per Execution field, enter the maximum number of lines from a summarized data file that a map/reduce script can process per execution.

      The system processes the summarized files from child section sources to generate the parent section source's file. If the number of lines in the summarized file surpasses the stipulated limit, the system processes the remaining lines in subsequent map/reduce executions.

  10. Click Save.

    NetSuite saves the processing parameters for the section source.

    Tip:

    If you need to revert your changes, refer to the system notes of the section source to verify the old field values. Take note of the values and enter them in the respective fields. If the field was previously empty, clear it.

    To open the system notes, click the Notes subtab and then System Notes. For more information, see System Notes Overview.

Related Topics

General Notices