Day 11 - Fine-Tuning Neural Network Hyperparameters

• Examples of hyperparameters
  • Number of hidden layers
  • Number of neurons per hidden layer
  • Learning rate
  • Batch size
  • Optimizer

• Hyperparameter tuning can be done by
  • Wrapping Keras model in object that mimics Scikit-Learn models (using keras.wrappers.scikit_learn.KerasRegressor and keras.wrappers.scikit_learn.KerasClassifier) and using grid search or randomized search from Scikit-Learn.
  • Using specialized libraries for neural network hyperparameter tuning like Hyperopt, Hyperas (based on Hyperopt), Keras Tuner, etc.

Tuning hyperparameters using Scikit-Learn

• Create a function that receives a set of hyperparameters, builds and compiles a Keras model.

• Set reasonable default values for the hyperparameters.

• Pass the function to keras.wrappers.scikit_learn.KerasRegressor or keras.wrappers.scikit_learn.KerasClassifier to create a model compatible with Scikit-Learn.

• The wrapped model can be used like a regular Scikit-Learn model: using fit(), score() and predict().

• Any extra parameters passed to the fit() method will be passed to the underlying Keras model.

• The score will be the opposite of MSE because Scikit-Learn works with scores, not losses (higher should be better).

Google Colaboratory

https://colab.research.google.com/drive/19vPJOXVEgdqiYeU9-vc_HFHJwByQ9G_t?usp=sharing