A single train/test split is noisy on small data. Cross-validation rotates which fold is test. cross_val_score returns one score per fold. The mean is a stabler estimate.

Goal

Run 5-fold CV, print each fold and the mean, then try a KFold splitter.

cross_val_score

from sklearn.datasets import make_classification
from sklearn.model_selection import cross_val_score
from sklearn.linear_model import LogisticRegression

X, y = make_classification(n_samples=200, n_features=4, random_state=0)
clf = LogisticRegression(max_iter=200)
scores = cross_val_score(clf, X, y, cv=5, scoring="accuracy")
print("folds", scores.round(3))
print("mean", round(scores.mean(), 3), "std", round(scores.std(), 3))

cv=5 means five folds. sklearn clones the estimator each time — your clf is not left fitted on the last fold unless you fit it yourself afterward.

Stratified folds

from sklearn.datasets import make_classification
from sklearn.model_selection import StratifiedKFold, cross_val_score
from sklearn.linear_model import LogisticRegression

X, y = make_classification(
    n_samples=200, n_features=4, weights=[0.8, 0.2], random_state=0
)
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=0)
clf = LogisticRegression(max_iter=200)
scores = cross_val_score(clf, X, y, cv=cv, scoring="accuracy")
print("folds", scores.round(3))
print("mean", round(scores.mean(), 3))

For classifiers, cv=5 already uses stratified folds. Passing StratifiedKFold lets you set shuffle and random_state.

Pipeline + CV

from sklearn.datasets import make_classification
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

X, y = make_classification(n_samples=200, n_features=4, random_state=0)
pipe = Pipeline(
    [
        ("scale", StandardScaler()),
        ("clf", LogisticRegression(max_iter=200)),
    ]
)
scores = cross_val_score(pipe, X, y, cv=5)
print("mean", round(scores.mean(), 3))

Each fold fits the scaler on that fold’s train rows only. That is why the scaler belongs in the pipeline.

Pitfall

Do not fit a scaler on all of X and then run CV on the scaled array — every fold would have seen the held-out rows. Nest the scaler in the pipeline.