Finished 'Tensorflow 101'

Hidden Layer

adding a hidden layer

H = tf.keras.layers.Dense(5, activation='swish')(X)

adding multiple hidden layers

H = tf.keras.layers.Dense(5, activation='swish')(X)
H = tf.keras.layers.Dense(3, activation='swish')(H)
H = tf.keras.layers.Dense(3, activation='swish')(H)

BE CAREFUL
(X) becomes (H) in the output when you add hidden layer(s)

Y = tf.keras.layers.Dense(1)(H)

make sure you check your model to see if hidden layers are added:

model.summary()

Appendix 1 - Tip regarding data

1. When onehot-encoding does not work
   When the data in the dependent variable is in written in numbers whereas the data are actually categorical, you need to check the type of the data and change it to 'category' by

   print(data.dtypes)
   data['nameofcolumn'] = data['nameofcolumn'].astype('category')

   cf. if you want to change categorical data into numerical data:

   data['nameofcolumn'] = data['nameofcolumn'].astype('int')

   or

   data['nameofcolumn'] = data['nameofcolumn'].astype('float')

2. When there is a NA(NaN) data in the csv file
   When there is an empty data in the file and you're trying to make a model with it, an error will occur.
   -> You would have to either add a number(replace NA with a number), or delete the row that contains NA

   to check if there is an NA:

   data.isna().sum()

   to fill in a certain number in NA:

   data['nameofcolumn'].fillna(certainnumber)

   to fill in the mean in NA:

   mean = data['nameofcolumn'].mean()
   print(mean)
   data['nameofcolumn'].fillna(mean)

Appendix 1 - Tip regarding model

To make a better learning model - we can use 'batch normalization' by separating the layer from activation and adding batch normalization inbetween.

• regression

  X = tf.keras.layers.Input(shape=[13])
  
  H = tf.keras.layers.Dense(8)(X)
  H = tf.keras.layers.BatchNormalization()(H)
  H = tf.keras.layers.Activation('swish')(H)
  
  H = tf.keras.layers.Dense(8)(H)
  H = tf.keras.layers.BatchNormalization()(H)
  H = tf.keras.layers.Activation('swish')(H)
  
  H = tf.keras.layers.Dense(8)(H)
  H = tf.keras.layers.BatchNormalization()(H)
  H = tf.keras.layers.Activation('swish')(H)
  
  Y = tf.keras.layers.Dense(1)(H)
  model = tf.keras.models.Model(X, Y)
  model.compile(loss='mse')

• classification

  X = tf.keras.layers.Input(shape=[13])
  
  H = tf.keras.layers.Dense(8)(X)
  H = tf.keras.layers.BatchNormalization()(H)
  H = tf.keras.layers.Activation('swish')(H)
  
  H = tf.keras.layers.Dense(8)(H)
  H = tf.keras.layers.BatchNormalization()(H)
  H = tf.keras.layers.Activation('swish')(H)
  
  H = tf.keras.layers.Dense(8)(H)
  H = tf.keras.layers.BatchNormalization()(H)
  H = tf.keras.layers.Activation('swish')(H)
  
  Y = tf.keras.layers.Dense(1)(H)
  model = tf.keras.models.Model(X, Y)
  model.compile(loss='mse')