How Many Trees in a Random Forest? A Revisited Approach With Plateau Search and Optuna Integration

Q: Is the scikit-learn default of 100 trees enough?

Sometimes, but it is a default rather than a decision. Whether 100 trees suffice is what the plateau criterion tests, and the sufficient size also depends on the other hyperparameters being tuned at the same time.

Q: What range should I give Optuna for n_estimators?

With plateau search, none. You give a starting size, a scale factor, a relative tolerance and a safety bound; the number of trees is never sampled from a predefined range.

Porvatov, Vadim A.; Dukhovny, Andrey A.; Lange, Andrey M.

doi:10.1109/ACCESS.2026.3705574

How Many Trees in a Random Forest?

Stop guessing the n_estimators range. PLATEAU finds where Random Forest performance flattens while Optuna tunes the rest — the open-source implementation accompanying the IEEE Access 2026 paper.

Quickstart (2 min) PyPI Conda-forge Code on GitHub Paper (IEEE Access) Preprint (arXiv) Theory (arXiv)

Stop guessing the `n_estimators` range

A Random Forest does not give HPO a useful interior optimum in the number of trees. The score rises, then plateaus. Once you are on that plateau, a fixed-range search is mostly choosing among nearly indistinguishable forests, so the reported tree count becomes tied to the upper bound you typed in.

Set the bound too low and the forest may be insufficient. Set it too high and you pay for trees that no longer move the score.

Search for the plateau, not the boundary

PLATEAU does not search for the best tree count inside a range. It searches for the plateau itself. For a current ensemble size B and scale factor sf, it evaluates the geometric triplet B/sf, B, B*sf. Relative OOB score changes move that triplet left, keep it in place, or move it right while Optuna/TPE tunes the remaining Random Forest hyperparameters.

You choose a relative tolerance ε (delta) — the score change you are willing to call negligible — instead of a [T_min, T_max] range you cannot justify.

See it in two minutes

The quickstart notebook uses the built-in Breast Cancer dataset to plot the OOB plateau, run the same TPE search under two different upper bounds, and then watch the PLATEAU triplet walk. Open it in Colab.

Reference tutorial run: TPE on [100, 2565] selects 688 trees at 0.9909 OOB AUC; PLATEAU selects 30 at 0.9899 and grows about 12× fewer trees while tuning. One dataset, one seed — the paper contains the benchmark.

Install and run

pip install rf-plateau-hpo

or with conda-forge:

conda install -c conda-forge rf-plateau-hpo

from rf_plateau_hpo import tune_rf_oob_plateau
from sklearn.datasets import load_breast_cancer
from sklearn.metrics import roc_auc_score

X, y = load_breast_cancer(return_X_y=True)
auc = lambda y_true, proba: roc_auc_score(y_true, proba[:, 1])

model, best_n_estimators, study, plateau_found = tune_rf_oob_plateau(
    X, y, problem="clf", score_func=auc, greater_is_better=True,
    n_estimators_start=100, scale_factor=1.5, delta=1e-3,
    max_trees=100000, n_trials=20, random_state=42, n_jobs=-1,
)
print(best_n_estimators)   # near-minimal sufficient number of trees

Frequently asked questions

How many trees does a Random Forest need?

There is no universal number. It depends on the dataset, the scoring metric, the other hyperparameters, and on how much score change you are willing to ignore. PLATEAU search makes that last quantity explicit and seeks a near-minimal sufficient ensemble size at that tolerance.

Is the scikit-learn default of 100 trees enough?

Sometimes — but it is a default, not a decision. Whether 100 trees suffice is exactly what the plateau criterion tests, and the sufficient size also depends on the other hyperparameters being tuned at the same time.

Can too many trees make a Random Forest overfit?

Adding trees mainly reduces variance rather than increasing capacity, so the score rises and then flattens. More trees are usually not the problem. Paying for trees that no longer move the score is. The bill shows up in training time, memory, and inference latency.

What range should I give Optuna for `n_estimators`?

With PLATEAU search, none. You give a starting size, a scale factor, a tolerance, and a safety bound; the number of trees is never sampled from a range.

How is this different from out-of-bag early stopping?

Early stopping reads a single score curve and can be sensitive to noise and prone to premature stopping. PLATEAU search accumulates information across optimization trials and uses a triplet of forest sizes, so the tree count and the other hyperparameters adapt to each other.

Papers

V. A. Porvatov, A. A. Dukhovny, and A. M. Lange, “How Many Trees in a Random Forest? A Revisited Approach With Plateau Search and Optuna Integration,” IEEE Access, vol. 14, pp. 93670–93693, 2026. doi:10.1109/ACCESS.2026.3705574 · arXiv:2606.03549

A. A. Dukhovny and A. M. Lange, “A Stationary-Distribution Theory for Triplet-Based Plateau Search in Random Forest Ensemble-Size Selection,” 2026. arXiv:2606.30837

Related work

The question goes back to Breiman (2001) and was studied empirically by Oshiro, Perez and Baranauskas (2012), Latinne et al. (2001), Hernández-Lobato et al. (2013), and Probst, Wright and Boulesteix (2019). These works study forest size and related Random Forest tuning questions, typically separately from joint adaptive ensemble-size selection within HPO. The contribution here is to treat ensemble size and the other Random Forest hyperparameters as interdependent and to resolve both inside a single Optuna run.