Neural Network Model

6.14 Neural Network Model

The ore.odmNN class creates a Neural Network (NN) model for classification and regression. The Neural Network models can be used to capture intricate nonlinear relationships between inputs and outputs or to find patterns in data.

The ore.odmNN class methods build a feed-forward neural network for regression and classification on OML4R proxy data frames. It supports multiple hidden layers, each with their own number of nodes and activation functions.

Each layer can have one of the following activation functions.

NNET_ACTIVATIONS_ARCTAN
NNET_ACTIVATIONS_BIPOLAR_SIG
NNET_ACTIVATIONS_LINEAR
NNET_ACTIVATIONS_LOG_SIG
NNET_ACTIVATIONS_RELU
NNET_ACTIVATIONS_TANH

The output layer is a single numeric or binary categorical target. The output layer can have any of the activation functions. It has the linear activation function by default.

Modeling with the ore.odmNN class is well-suited for noisy and complex data such as sensor data. Problems that such data might have are the following:

Potentially many (numeric) predictors, for example, pixel values
The target may be discrete-valued, real-valued, or a vector of such values
Training data may contain errors – robust to noise
Fast scoring
Model transparency is not required; models difficult to interpret

Typical steps in Neural Network modeling are the following:

Specifying the architecture
Preparing the data
Building the model
Specifying the stopping criteria: iterations, error on a validation set within tolerance
Viewing statistical results from the model
Improving the model

Settings for a Neural Network Model

The following table lists settings for NN models.

Table 6-13 Neural Network Model Settings

Setting Name	Setting Value	Description
`NNET_HIDDEN_LAYERS`	`X >= 0`	Defines the topology by number of hidden layers. The default value is `1`.
`NNET_NODES_PER_LAYER`	A list of positive integers	Defines the topology by number of nodes per layer. Different layers can have different number of nodes. The value should be non-negative integers and comma separated. For example, '10, 20, 5'. The setting values must be consistent with `NNET_HIDDEN_LAYERS`. The default number of nodes per layer is the number of attributes or `50` (if the number of attributes > `50`).
`NNET_ACTIVATIONS`	A list of the following strings: `"NNET_ACTIVATIONS_LOG_SIG"` `"NNET_ACTIVATIONS_LINEAR"` `"NNET_ACTIVATIONS_TANH"` `"NNET_ACTIVATIONS_ARCTAN"` `"NNET_ACTIVATIONS_BIPOLAR_SIG"`	Defines the activation function for the hidden layers. For example, '''`NNET_ACTIVATIONS_BIPOLAR_SIG`'', ''`NNET_ACTIVATIONS_TANH`'''. Different layers can have different activation functions. The default value is `"NNET_ACTIVATIONS_LOG_SIG"`. The number of activation functions must be consistent with `NNET_HIDDEN_LAYERS` and `NNET_NODES_PER_LAYER` . Note: All quotes are single and two single quotes are used to escape a single quote in SQL statements.
`NNET_WEIGHT_LOWER_BOUND`	`A real number`	The setting specifies the lower bound of the region where weights are randomly initialized. `NNET_WEIGHT_LOWER_BOUND` and `NNET_WEIGHT_UPPER_BOUND` must be set together. Setting one and not setting the other raises an error. `NNET_WEIGHT_LOWER_BOUND` must not be greater than `NNET_WEIGHT_UPPER_BOUND`. The default value is `–sqrt(6/(l_nodes+r_nodes))`. The value of `l_nodes` for: input layer dense attributes is (`1+number of dense attributes`) input layer sparse attributes is `number of sparse attributes` each hidden layer is (`1+number of nodes in that hidden layer`) The value of `r_nodes` is the number of nodes in the layer that the weight is connecting to.
`NNET_WEIGHT_UPPER_BOUND`	`A real number`	This setting specifies the upper bound of the region where weights are initialized. It should be set in pairs with `NNET_WEIGHT_LOWER_BOUND` and its value must not be smaller than the value of `NNET_WEIGHT_LOWER_BOUND`. If not specified, the values of `NNET_WEIGHT_LOWER_BOUND` and `NNET_WEIGHT_UPPER_BOUND` are system determined. The default value is `sqrt(6/(l_nodes+r_nodes))`. See `NNET_WEIGHT_LOWER_BOUND`.
`NNET_ITERATIONS`	`X > 0`	This setting specifies the maximum number of iterations in the Neural Network algorithm. The default value is `200`.
`NNET_TOLERANCE`	`0 < X < 1`	Defines the convergence tolerance setting of the Neural Network algorithm. The default value is `0.000001`.
`NNET_REGULARIZER`	`NNET_REGULARIZER_NONE` `NNET_REGULARIZER_L2` `NNET_REGULARIZER_HELDASIDE`	Regularization setting for Neural Network algorithm. If the total number of training rows is greater than 50000, the default is `NNET_REGULARIZER_HELDASIDE`. If the total number of training rows is less than or equal to 50000, the default is `NNET_REGULARIZER_NONE`.
`NNET_HELDASIDE_RATIO`	`0 <= X <= 1`	Define the held ratio for the held-aside method. The default value is `0.25`.
`NNET_HELDASIDE_MAX_FAIL`	The value must be a positive integer.	With `NNET_REGULARIZER_HELDASIDE`, the training process is stopped early if the network performance on the validation data fails to improve or remains the same for `NNET_HELDASIDE_MAX_FAIL` epochs in a row. The default value is `6`
`NNET_REG_LAMBDA`	`TO_CHAR(X >= 0)`	Defines the L2 regularization parameter lambda. This can not be set together with `NNET_REGULARIZER_HELDASIDE`. The default value is `1`.

Example 6-16 Building a Neural Network Model

This example creates an NN model and uses some of the methods of the ore.odmNN class.


# Turn off row ordering warnings

options(ore.warn.order=FALSE)

# Data setup

set.seed(7654)
x <- seq(0.1, 5, by = 0.02)
weights <- round(rnorm(length(x),10,3))
y <- log(x) + rnorm(x, sd = 0.2)

# Create a temporary OML4R proxy object DAT.

DAT <- ore.push(data.frame(x=x, y=y, weights=weights))

# Create an NN regression model object. Fit the NN model according to the data and setting parameters.

mod.nn <- ore.odmNN(y~x, DAT,"regression",
                          odm.settings = list(nnet_hidden_layers = 1))
weight(mod.nn)
summary(mod.nn)

# Use the model to make predictions on the input data.

pred.nn <- predict(mod.nn, DAT, "y")
head(pred.nn, 10)

Listing for This Example

Table 6-14 A data.frame: 4 x 6

LAYER	IDX_FROM	IDX_TO	ATTRIBUTE_NAME	ATTRIBUTE_VALUE	WEIGHT
<dbl>	<dbl>	<dbl>	<chr>	<chr>	<dbl>
0	0	0	x	NA	-1.0663866
0	NA	0	NA	NA	-7.4897304
1	0	0	NA	NA	-1068.0117188
1	NA	0	NA	NA	0.9961451

Table 6-15 A data.frame: 10 x 2

y	PREDICTION
<dbl>	<dbl>
-2.376195	-1.648826
-1.906485	-1.601597
-2.027240	-1.555065
-1.541951	-1.509221
-1.654645	-1.464055
-1.742211	-1.419556
-1.320646	-1.375714
-1.357442	-1.332520
-1.442755	-1.289965
-1.192586	-1.248039

Example 6-17 ore.odmNN classification

This example creates an NN model and uses some of the methods of the ore.odmNN classification class.


# Turn off row ordering warnings

options(ore.warn.order=FALSE)

# Data setup

m <- mtcars
m$gear <- as.factor(m$gear)
m$cyl  <- as.factor(m$cyl)
m$vs   <- as.factor(m$vs)
m$ID   <- 1:nrow(m)

# Create a temporary OML4R proxy object for the MTCARS table.

MTCARS <- ore.push(m)
row.names(MTCARS) <- MTCARS$ID

# Create an NN classification model object. Fit the NN model according to the data and setting parameters.

mod.nn  <- ore.odmNN(gear ~ ., MTCARS,"classification",
                         odm.settings = list(nnet_hidden_layers = 2,
                                             nnet_activations = c("'NNET_ACTIVATIONS_LOG_SIG'", "'NNET_ACTIVATIONS_TANH'"),
                                             nnet_nodes_per_layer = c(5, 2)))
head(weight(mod.nn), 10)

# Use the model to make predictions on the input data.

pred.nn <- predict(mod.nn, MTCARS, "gear")

# Generate a confusion matrix.
with(pred.nn, table(gear, PREDICTION))

Listing for This Example

Table 6-16 A data.frame: 10 x 6

LAYER	IDX_FROM	IDX_TO	ATTRIBUTE_NAME	ATTRIBUTE_VALUE	WEIGHT
<dbl>	<dbl>	<dbl>	<chr>	<chr>	<dbl>
0	0	0	ID	NA	12.424586
0	0	1	ID	NA	-9.953163
0	0	2	ID	NA	-7.516252
0	0	3	ID	NA	-1.100170
0	0	4	ID	NA	-15.955383
0	1	0	am	NA	21.585514
0	1	1	am	NA	-3.228476
0	1	2	am	NA	-22.794853
0	1	3	am	NA	15.349457
0	1	4	am	NA	-19.099138

Parent topic: OML4R Classes That Provide Access to In-Database Machine Learning Algorithms