Random Forest Hyper-parameters

Random Forest Classifier Hyperparameters Overview
📌 The video focuses on understanding the hyperparameters of the Random Forest Classifier algorithm, noting that many are shared with Random Forest Regression.
⚙️ The hyperparameters are broadly categorized into three groups: those specific to the overall Random Forest, those used for training individual Decision Trees, and those common to all machine learning algorithms.
📊 The speaker introduced a visualization tool showing four primary RF tuning parameters: `n_estimators`, `max_features`, `bootstrap`, and `max_samples`.

Core Random Forest Hyperparameters
🌳 `n_estimators`: Determines the number of individual trees used to form the forest; the default value is 100.
📏 `max_features`: Controls the number of features considered when looking for the best split in a tree, with options including 'auto' ($\sqrt{\text{features}}$), 'sqrt', 'log2', or a specific integer/float percentage (between 0 and 1).
🔀 `bootstrap`: A Boolean defining whether bootstrap samples (sampling with replacement) are used when building trees; the default is True.
🔢 `max_samples`: Specifies the number of samples (rows) drawn from the dataset to train each individual tree; keeping this value between 50% and 75% yielded the most optimal results in the demonstration.

Individual Decision Tree Parameters (Briefly Mentioned)
✂️ Parameters governing individual tree growth include `criterion` (e.g., Gini or Entropy), `max_depth`, `min_samples_split`, and `min_samples_leaf`.
📚 For detailed explanations of these tree-specific parameters, viewers are directed to another video by the speaker that covers them in depth using an existing tool.

Advanced and Miscellaneous Parameters
⚡ `n_jobs`: Utilized for speeding up training by allowing the use of multiple CPU cores for parallel processing.
🎲 `random_state`: Used to ensure reproducibility by setting a constant seed for the random feature selection process during training.
⚖️ `class_weight`: Can be passed as 'balanced' or a custom dictionary to handle imbalanced datasets by adjusting class weights.

Key Points & Insights
➡️ Increasing the `n_estimators` generally improves accuracy but increases training time; overfitting boundaries tend to smooth out as the number of trees increases.
➡️ Experimenting with `max_samples` is crucial; very low or very high values (e.g., below 50% or above 75%) resulted in significantly worse performance in the demonstration.
➡️ For `max_features`, using the default ('auto' or $\sqrt{\text{features}}$) resulted in decent accuracy, but setting it explicitly to a value near 200 led to slightly better results in the test case.
➡️ For Random Forest Regression, parameters are largely the same, but the `criterion` typically defaults to MSE (Mean Squared Error) instead of Gini/Entropy.

📸 Video summarized with SummaryTube.com on Nov 27, 2025, 10:42 UTC