9.3 Parameters and Hyperparameters

A model’s parameters are learned through the model fitting process.  In a logistic regression model, for instance, parameters include the intercept and the coefficients associated with each term.  

Hyperparameters, however, are set by the modeler.   Like the knobs on a stereo that can adjust things like volume, treble, and bass, hyperparameters can be adjusted, or “tuned” to suit some particular purpose.  

When using scikit-learn for any tree model implementation, remember that some hyperparameter adjustment is likely to be needed.  By default, scikit-learn instructs tree models to grow to the point that their terminal nodes, or leaf nodes, are completely homogeneous.  This is almost certain to cause overfitting.  

In the section below, we demonstrate the hyperparameter tuning process for a random forest model built with the nyc_historical dataset.