LearningAlgorithmBeginner-friendly
Decision Trees — Learn Splits & Impurity
Learn decision trees: splits, Gini/entropy, overfitting, pruning, and interpretability.
What you'll learn
- How trees split data into simple if/else rules.
- What impurity (Gini/entropy) measures.
- Why trees overfit and how to control it.
Intuition: ask the best question next
A decision tree repeatedly chooses a feature + threshold that best separates the data.
Each internal node is a question (e.g., “is age > 30?”) and leaves output a prediction.
Impurity and information gain
Gini and entropy quantify how mixed the labels are in a node.
A good split reduces impurity a lot — meaning each child node is more “pure.”
Overfitting + pruning
Trees can memorize noise by growing deep with many leaves.
Control depth, minimum samples per leaf, or prune after training to improve generalization.
Key takeaways
- Very interpretable rules, minimal preprocessing.
- Can overfit easily without constraints.
- Works well with non-linear relationships.
- Ensembles (RF/XGBoost) often outperform a single tree.
Want more ML topics added here (SVM, Naive Bayes, CNN, PCA, Decision Trees)?
Browse Machine Learning ->