The Data we'll be modeling

Please first and foremost be aware that an R widget allows one to access any data available within SAC. This means that you can train a neural network using a data set or a data model already imported into the SAC environment previously.

For the purpose of this blogpost I'm using the data set showcasing a highly non-linear relationship between two variables (X and Y), as depicted in the screenshot below.

The dependent variable is the Y variable in this context (Y axis) and the X is the independent variable, which we will be using as a predictor in our model.

One can clearly observe the sheer necessity for non-linear modeling technique for these data, as i.e. a simple linear regression (straight line with an intercept and a slope) would definitely be inappropriate to model this kind of complexity:  

Please feel free to right click the image and open it separately in another window or tab to see a bigger picture... :-)

The code to generate this graphic in the R widget is listed below:

----

# 'dta' is a data set, which contains these data.

require(plotly)

fig <- plot_ly(dta, x = ~x, y = ~y, color = I("orange2"))

fig <- fig %>% add_markers()

fig

----

In general this data set contains 2 000 observations of two variables (X and Y).

The next step would be to split it into training and test sets. The test set will not be used for modeling purpose at all, but only to assess the model's performance afterwards.

The training set will itself be splitted into the actual data, which will be used for the training of the neural network and the so called evaluation set, which is necessary to avoid overfitting of the underlying training set.

The code to produce these two plots within two separate R widgets is listed below:

----

# Training data plot R widget graphic #

require(plotly)

set.seed(42)
test_ids <- sample(1 : nrow(dta), size = 300, replace = FALSE)
test_set <- dta[test_ids, ]
train_set <- dta[-test_ids, ]

x_mean <- mean(train_set$x); x_sd <- sd(train_set$x)
y_mean <- mean(train_set$y); y_sd <- sd(train_set$y)

train_set$x <- (train_set$x - x_mean) / x_sd
train_set$y <- (train_set$y - y_mean) / y_sd

fig <- plot_ly(train_set, x = ~x, y = ~y, color = I("orange2"))
fig <- fig %>% add_markers()
fig

----

# Test data plot R widget graphic #

require(plotly)

set.seed(42)
test_ids <- sample(1 : nrow(dta), size = 300, replace = FALSE)
test_set <- dta[test_ids, ]
train_set <- dta[-test_ids, ]

x_mean <- mean(train_set$x); x_sd <- sd(train_set$x)
y_mean <- mean(train_set$y); y_sd <- sd(train_set$y)

test_set$x <- (test_set$x - x_mean) / x_sd
test_set$y <- (test_set$y - y_mean) / y_sd

fig <- plot_ly(test_set, x = ~x, y = ~y, color = I("forestgreen"))
fig <- fig %>% add_markers()
fig
 

----

Training of the neural network model

As of right now, we are ready to train the neural network model completely within the SAP Analytics Cloud infrastructure in an R widget framework.

For this purpose the evaluation data set has been derived from the training data and used to monitor the progress of the neural network training.

You can see the semantics of this evaluation set and the training progress loop in the code snippet below:

----

require(nnet)

set.seed(42)
test_ids <- sample(1 : nrow(dta), size = 300, replace = FALSE)
test_set <- dta[test_ids, ]
train_set <- dta[-test_ids, ]

x_mean <- mean(train_set$x); x_sd <- sd(train_set$x)
y_mean <- mean(train_set$y); y_sd <- sd(train_set$y)

train_set$x <- (train_set$x - x_mean) / x_sd
train_set$y <- (train_set$y - y_mean) / y_sd

test_set$x <- (test_set$x - x_mean) / x_sd
test_set$y <- (test_set$y - y_mean) / y_sd

set.seed(42)
eval_ids <- sample(1 : nrow(train_set), size = 500, replace = FALSE)
eval_set <- train_set[eval_ids, ]
train_set <- train_set[-eval_ids, ]

metrics <- data.frame()

for (i in 1 : 100) {
      set.seed(1337)
      nn <- nnet(y ~ x, data = train_set, size = 40, maxit = i, linout = TRUE, rang = 0.7)
      RMSE_train <- sqrt(mean((train_set$y - predict(nn))^2))
      RMSE_eval  <- sqrt(mean((eval_set$y  - predict(nn, newdata = eval_set))^2))
      metrics <- rbind(metrics, data.frame(RMSE_train, RMSE_eval))
}

plot(1 : nrow(metrics), metrics$RMSE_train, type = "l", col = "green2", lwd = 2, 
     main = "Training vs. Evaluation RMSE", xlab = "Iteration", ylab = "RMSE", las = 1)
lines(1 : nrow(metrics), metrics$RMSE_eval, col = "red", lwd = 2, xpd = TRUE)
legend("topright", legend = c("train", "eval"), fill = c("green2", "red"), border = NA, bty = "n")
abline(v = which.min(metrics$RMSE_eval), col = "steelblue4", lty = 2)
title(sub = which.min(metrics$RMSE_eval))
 

----

The corresponding result of this process is depicted in the plot below. Please note that the 49th iteration of this fully connected feed forward neural network presents the minimum for the loss function on the evaluation data set and, thus, should be considered as a pivotal point of the training progress.

The training should be stopped at this particular iteration.

This technique is called cross-validation and serves as a means to avoid overfitting of the underlying training data set:

One can clearly see that the severe non-linearity has been captured adequately by the neural network model and the used cross-validation did indeed prohibit the model to be overfitted on the training data set, thus giving a robust and obviously very coherent and adequate predictions for these never seen before data set observations.

Final thoughts

Neural Networks are not new on the field of statistical modeling and research. They have been around for already quite some time now.

Despite their power in predicting the data, they are quite cumbersome to deal with when it comes to explaining the relationship(s) between the outcome and the predictor(s).

Please use this technique with utmost care and do not exaggerate its usefulness.

Sometimes it's better and more versatile to use a simpler modeling approach if no complex structure is observed in the data to be modeled.

However, if the underlying relationship's behaviour between the outcome and the predictors is prone to the highly non-linear pattern structures, - the neural network can potentially be a plausible solution for the modeling.

Please be sure to use the cross-validation technique to avoid overfitting of the actual training data. 

All things considered: Have fun while modeling!