Use Multiple Threads and Multiple Queues to Run CSV Import Jobs
In accounts that have purchased one or more SuiteCloud Plus licenses, the Import Assistant includes two advanced import options that you can set to improve import performance.
For a summary of SuiteCloud Plus capabilities, see SuiteCloud Plus Settings.
-
You can check the Use Multi-threading box, to use multiple threads instead of a single thread to process an import job. See Using Multiple Threads.
-
You can set a queue number from 1 to 5, to spread the processing of import jobs across multiple queues, instead of using a single queue for all jobs. See Using Multiple Queues.
These options are available on the step two Import Options page of the Import Assistant, under Advanced Options. For instructions for completing them, see Set Advanced CSV Import Options.
In addition to supporting import job performance enhancement, the SuiteCloud Plus license also supports access to more concurrent web services sessions and to multiple processors for scheduled scripts and map/reduce scripts. To learn more about SuiteCloud Plus, or to purchase this add-on module, contact your NetSuite account manager.
Using Multiple Threads
By default, a single thread is used for processing CSV import jobs, and each CSV file row is submitted in order. In accounts that have purchased a SuiteCloud Plus license, an option is available to use multiple threads. If this option is enabled for an import job, processing is optimized across multiple threads for best performance, and the order in which CSV file rows are submitted cannot be predicted. Accordingly, you should only enable multi-threading for an import job, if the order in which rows are submitted does not matter, meaning later CSV file row data is not dependent on any earlier row data.
If you enable this option for an import job where order does matter, some records may fail to import, because they are dependent on records which have yet to be imported. When this situation occurs, you can rerun the job a second time to import the dependent records.
If you purchase one SuiteCloud Plus license, two threads are available for each import job. In some cases, you may be able to purchase three, six, or 12 licenses to support 10 threads per job.
For more information, see SuiteCloud Plus Settings.
For information about service tiers, see NetSuite Service Tiers.
Using Multiple Queues
By default, all CSV import jobs use a single queue. This queue is defined as queue 1. In accounts that have purchased a SuiteCloud Plus license, five queues are available for import jobs, instead of a single queue. The queue to be used to process each import job can be specified in a dropdown list in the Import Assistant. This ability allows the division of import job processing across multiple queues, enhancing performance.
There are never more than five import job queues regardless of how many SuiteCloud Plus licenses are purchased.
Import jobs in each queue execute serially. For example, if there are two import jobs in queue 1, the first import job must complete before the second import job begins. Across all five queues, the import jobs execute concurrently. For example, if there is one import job in each of the five queues, all import jobs run at the same time.
Should you choose to purchase the SuiteCloud Plus add-on module, be aware that all of your saved import jobs are, by default, initially placed in queue 1. You need to go the step two Import Options page of each import job and assign a different queue as needed. Also note that if you do not assign a queue number to a new import job, the job is automatically assigned to queue 1.
A Queue column displays on the import Job Status page to indicate which queue is processing the job. Users can set the Queue filter to sort import jobs based on queue number.