bhylu.com

Advertisements

Cross Validation

Cross-validation repeats train/validation splits to assess a modelling choice. It gives more information than one convenient split, but it still depends on the split assumptions.

Understand the idea

Stratified folds preserve class proportions approximately for this classification demo. Grouped or temporal data needs an appropriate splitter instead. Pipeline preprocessing so every fold learns transformations only from its training rows. Cross-validation used to select a model is not the same as an untouched final test evaluation.

Where to try it

Run locally with scikit-learn and NumPy installed in your virtual environment (py -m pip install scikit-learn numpy). These examples are executed during course review, but the online runner does not currently preload scikit-learn. The small data is for learning the API, not evidence of production model quality.

A worked example

Python

from sklearn.model_selection import StratifiedKFold, cross_val_score
from sklearn.tree import DecisionTreeClassifier
X = [[value] for value in range(18)]
y = [0] * 9 + [1] * 9
folds = StratifiedKFold(3, shuffle=True, random_state=7)
scores = cross_val_score(DecisionTreeClassifier(max_depth=1, random_state=7), X, y, cv=folds)
print(scores.round(3).tolist())

Read the result

The output lists one accuracy per validation fold. Differences reflect which rows each fold withholds in this tiny sample.

Try one change

Report both mean and spread, then explain why the sample is too small for a strong deployment claim.

For package options, see the scikit-learn documentation.