MySQL 8.0 Reference Manual Including MySQL NDB Cluster 8.0

25.5.13 ndb_import — Import CSV Data Into NDB

ndb_import imports CSV-formatted data, such as that produced by mysqldump --tab, directly into NDB using the NDB API. ndb_import requires a connection to an NDB management server (ndb_mgmd) to function; it does not require a connection to a MySQL Server.

Usage

ndb_import db_name file_name options

ndb_import requires two arguments. db_name is the name of the database where the table into which to import the data is found; file_name is the name of the CSV file from which to read the data; this must include the path to this file if it is not in the current directory. The name of the file must match that of the table; the file's extension, if any, is not taken into consideration. Options supported by ndb_import include those for specifying field separators, escapes, and line terminators, and are described later in this section.

Prior to NDB 8.0.30, ndb_import rejects any empty lines which it reads from the CSV file. Beginning with NDB 8.0.30, when importing a single column, an empty value that can be used as the column value, ndb_import handles it in the same manner as a LOAD DATA statement does.

ndb_import must be able to connect to an NDB Cluster management server; for this reason, there must be an unused [api] slot in the cluster config.ini file.

To duplicate an existing table that uses a different storage engine, such as InnoDB, as an NDB table, use the mysql client to perform a SELECT INTO OUTFILE statement to export the existing table to a CSV file, then to execute a CREATE TABLE LIKE statement to create a new table having the same structure as the existing table, then perform ALTER TABLE ... ENGINE=NDB on the new table; after this, from the system shell, invoke ndb_import to load the data into the new NDB table. For example, an existing InnoDB table named myinnodb_table in a database named myinnodb can be exported into an NDB table named myndb_table in a database named myndb as shown here, assuming that you are already logged in as a MySQL user with the appropriate privileges:

  1. In the mysql client:

    mysql> USE myinnodb;
    
    mysql> SELECT * INTO OUTFILE '/tmp/myndb_table.csv'
         >  FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' ESCAPED BY '\\'
         >  LINES TERMINATED BY '\n'
         >  FROM myinnodbtable;
    
    mysql> CREATE DATABASE myndb;
    
    mysql> USE myndb;
    
    mysql> CREATE TABLE myndb_table LIKE myinnodb.myinnodb_table;
    
    mysql> ALTER TABLE myndb_table ENGINE=NDB;
    
    mysql> EXIT;
    Bye
    $>
    

    Once the target database and table have been created, a running mysqld is no longer required. You can stop it using mysqladmin shutdown or another method before proceeding, if you wish.

  2. In the system shell:

    # if you are not already in the MySQL bin directory:
    $> cd path-to-mysql-bin-dir
    
    $> ndb_import myndb /tmp/myndb_table.csv --fields-optionally-enclosed-by='"' \
        --fields-terminated-by="," --fields-escaped-by='\\'
    

    The output should resemble what is shown here:

    job-1 import myndb.myndb_table from /tmp/myndb_table.csv
    job-1 [running] import myndb.myndb_table from /tmp/myndb_table.csv
    job-1 [success] import myndb.myndb_table from /tmp/myndb_table.csv
    job-1 imported 19984 rows in 0h0m9s at 2277 rows/s
    jobs summary: defined: 1 run: 1 with success: 1 with failure: 0
    $>
    

All options that can be used with ndb_import are shown in the following table. Additional descriptions follow the table.

Table 25.35 Command-line options used with the program ndb_import

Format Description Added, Deprecated, or Removed

--abort-on-error

Dump core on any fatal error; used for debugging

(Supported in all NDB releases based on MySQL 8.0)

--ai-increment=#

For table with hidden PK, specify autoincrement increment. See mysqld

(Supported in all NDB releases based on MySQL 8.0)

--ai-offset=#

For table with hidden PK, specify autoincrement offset. See mysqld

(Supported in all NDB releases based on MySQL 8.0)

--ai-prefetch-sz=#

For table with hidden PK, specify number of autoincrement values that are prefetched. See mysqld

(Supported in all NDB releases based on MySQL 8.0)

--character-sets-dir=path

Directory containing character sets

(Supported in all NDB releases based on MySQL 8.0)

--connect-retries=#

Number of times to retry connection before giving up

(Supported in all NDB releases based on MySQL 8.0)

--connect-retry-delay=#

Number of seconds to wait between attempts to contact management server

(Supported in all NDB releases based on MySQL 8.0)

--connect-string=connection_string,

-c connection_string

Same as --ndb-connectstring

(Supported in all NDB releases based on MySQL 8.0)

--connections=#

Number of cluster connections to create

(Supported in all NDB releases based on MySQL 8.0)

--continue

When job fails, continue to next job

(Supported in all NDB releases based on MySQL 8.0)

--core-file

Write core file on error; used in debugging

(Supported in all NDB releases based on MySQL 8.0)

--csvopt=opts

Shorthand option for setting typical CSV option values. See documentation for syntax and other information

(Supported in all NDB releases based on MySQL 8.0)

--db-workers=#

Number of threads, per data node, executing database operations

(Supported in all NDB releases based on MySQL 8.0)

--defaults-extra-file=path

Read given file after global files are read

(Supported in all NDB releases based on MySQL 8.0)

--defaults-file=path

Read default options from given file only

(Supported in all NDB releases based on MySQL 8.0)

--defaults-group-suffix=string

Also read groups with concat(group, suffix)

(Supported in all NDB releases based on MySQL 8.0)

--errins-type=name

Error insert type, for testing purposes; use "list" to obtain all possible values

(Supported in all NDB releases based on MySQL 8.0)

--errins-delay=#

Error insert delay in milliseconds; random variation is added

(Supported in all NDB releases based on MySQL 8.0)

--fields-enclosed-by=char

Same as FIELDS ENCLOSED BY option for LOAD DATA statements. For CSV input this is same as using --fields-optionally-enclosed-by

(Supported in all NDB releases based on MySQL 8.0)

--fields-escaped-by=char

Same as FIELDS ESCAPED BY option for LOAD DATA statements

(Supported in all NDB releases based on MySQL 8.0)

--fields-optionally-enclosed-by=char

Same as FIELDS OPTIONALLY ENCLOSED BY option for LOAD DATA statements

(Supported in all NDB releases based on MySQL 8.0)

--fields-terminated-by=char

Same as FIELDS TERMINATED BY option for LOAD DATA statements

(Supported in all NDB releases based on MySQL 8.0)

--help,

-?

Display help text and exit

(Supported in all NDB releases based on MySQL 8.0)

--idlesleep=#

Number of milliseconds to sleep waiting for more to do

(Supported in all NDB releases based on MySQL 8.0)

--idlespin=#

Number of times to retry before idlesleep

(Supported in all NDB releases based on MySQL 8.0)

--ignore-lines=#

Ignore first # lines in input file. Used to skip a non-data header

(Supported in all NDB releases based on MySQL 8.0)

--input-type=name

Input type: random or csv

(Supported in all NDB releases based on MySQL 8.0)

--input-workers=#

Number of threads processing input. Must be 2 or more if --input-type is csv

(Supported in all NDB releases based on MySQL 8.0)

--keep-state

State files (except non-empty *.rej files) are normally removed on job completion. Using this option causes all state files to be preserved instead

(Supported in all NDB releases based on MySQL 8.0)

--lines-terminated-by=char

Same as LINES TERMINATED BY option for LOAD DATA statements

(Supported in all NDB releases based on MySQL 8.0)

--login-path=path

Read given path from login file

(Supported in all NDB releases based on MySQL 8.0)

--max-rows=#

Import only this number of input data rows; default is 0, which imports all rows

(Supported in all NDB releases based on MySQL 8.0)

--missing-ai-column='name'

Indicates that auto-increment values are missing from CSV file to be imported.

ADDED: NDB 8.0.30

--monitor=#

Periodically print status of running job if something has changed (status, rejected rows, temporary errors). Value 0 disables. Value 1 prints any change seen. Higher values reduce status printing exponentially up to some pre-defined limit

(Supported in all NDB releases based on MySQL 8.0)

--ndb-connectstring=connection_string,

-c connection_string

Set connect string for connecting to ndb_mgmd. Syntax: "[nodeid=id;][host=]hostname[:port]". Overrides entries in NDB_CONNECTSTRING and my.cnf

(Supported in all NDB releases based on MySQL 8.0)

--ndb-mgmd-host=connection_string,

-c connection_string

Same as --ndb-connectstring

(Supported in all NDB releases based on MySQL 8.0)

--ndb-nodeid=#

Set node ID for this node, overriding any ID set by --ndb-connectstring

(Supported in all NDB releases based on MySQL 8.0)

--ndb-optimized-node-selection

Enable optimizations for selection of nodes for transactions. Enabled by default; use --skip-ndb-optimized-node-selection to disable

(Supported in all NDB releases based on MySQL 8.0)

--no-asynch

Run database operations as batches, in single transactions

(Supported in all NDB releases based on MySQL 8.0)

--no-defaults

Do not read default options from any option file other than login file

(Supported in all NDB releases based on MySQL 8.0)

--no-hint

Tells transaction coordinator not to use distribution key hint when selecting data node

(Supported in all NDB releases based on MySQL 8.0)

--opbatch=#

A db execution batch is a set of transactions and operations sent to NDB kernel. This option limits NDB operations (including blob operations) in a db execution batch. Therefore it also limits number of asynch transactions. Value 0 is not valid

(Supported in all NDB releases based on MySQL 8.0)

--opbytes=#

Limit bytes in execution batch (default 0 = no limit)

(Supported in all NDB releases based on MySQL 8.0)

--output-type=name

Output type: ndb is default, null used for testing

(Supported in all NDB releases based on MySQL 8.0)

--output-workers=#

Number of threads processing output or relaying database operations

(Supported in all NDB releases based on MySQL 8.0)

--pagesize=#

Align I/O buffers to given size

(Supported in all NDB releases based on MySQL 8.0)

--pagecnt=#

Size of I/O buffers as multiple of page size. CSV input worker allocates double-sized buffer

(Supported in all NDB releases based on MySQL 8.0)

--polltimeout=#

Timeout per poll for completed asynchonous transactions; polling continues until all polls are completed, or error occurs

(Supported in all NDB releases based on MySQL 8.0)

--print-defaults

Print program argument list and exit

(Supported in all NDB releases based on MySQL 8.0)

--rejects=#

Limit number of rejected rows (rows with permanent error) in data load. Default is 0 which means that any rejected row causes a fatal error. The row exceeding the limit is also added to *.rej

(Supported in all NDB releases based on MySQL 8.0)

--resume

If job aborted (temporary error, user interrupt), resume with rows not yet processed

(Supported in all NDB releases based on MySQL 8.0)

--rowbatch=#

Limit rows in row queues (default 0 = no limit); must be 1 or more if --input-type is random

(Supported in all NDB releases based on MySQL 8.0)

--rowbytes=#

Limit bytes in row queues (0 = no limit)

(Supported in all NDB releases based on MySQL 8.0)

--state-dir=path

Where to write state files; currect directory is default

(Supported in all NDB releases based on MySQL 8.0)

--stats

Save performance related options and internal statistics in *.sto and *.stt files. These files are kept on successful completion even if --keep-state is not used

(Supported in all NDB releases based on MySQL 8.0)

--table=name,

-t name

Name of target to import data into; default is base name of input file

ADDED: NDB 8.0.28

--tempdelay=#

Number of milliseconds to sleep between temporary errors

(Supported in all NDB releases based on MySQL 8.0)

--temperrors=#

Number of times a transaction can fail due to a temporary error, per execution batch; 0 means any temporary error is fatal. Such errors do not cause any rows to be written to .rej file

(Supported in all NDB releases based on MySQL 8.0)

--usage,

-?

Display help text and exit; same as --help

(Supported in all NDB releases based on MySQL 8.0)

--verbose[=#],

-v [#]

Enable verbose output

(Supported in all NDB releases based on MySQL 8.0)

--version,

-V

Display version information and exit

(Supported in all NDB releases based on MySQL 8.0)


As with LOAD DATA, options for field and line formatting much match those used to create the CSV file, whether this was done using SELECT INTO ... OUTFILE, or by some other means. There is no equivalent to the LOAD DATA statement STARTING WITH option.