predict returns a hard label. predict_proba returns a probability per class. The default threshold is 0.5. You can raise it when false positives are expensive.
Goal
Print class probabilities, then apply a custom threshold.
predict_proba
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
X, y = make_classification(n_samples=200, n_features=4, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.25, random_state=0
)
clf = LogisticRegression(max_iter=200)
clf.fit(X_train, y_train)
proba = clf.predict_proba(X_test)
print("classes", clf.classes_)
print("proba first 5:\\n", proba[:5].round(3))
print("predict first 5", clf.predict(X_test)[:5])Column i of proba is the probability of clf.classes_[i]. Rows sum to 1.
Threshold
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix
X, y = make_classification(n_samples=200, n_features=4, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.25, random_state=0
)
clf = LogisticRegression(max_iter=200)
clf.fit(X_train, y_train)
p1 = clf.predict_proba(X_test)[:, 1]
for thresh in (0.3, 0.5, 0.7):
pred = (p1 >= thresh).astype(int)
cm = confusion_matrix(y_test, pred)
print("thresh", thresh, "matrix", cm.ravel().tolist())cm.ravel() is [tn, fp, fn, tp] for two classes. A higher threshold predicts class 1 less often.
Pitfall
Not every estimator has predict_proba (some SVMs need probability=True). Check with hasattr(clf, "predict_proba").