A classifier predicts a class label. LogisticRegression is a linear model. KNeighborsClassifier votes among nearby training rows. Both share the same fit / predict API.

Goal

Fit logistic regression and k-nearest neighbors on the same split and compare accuracy.

Logistic regression

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

X, y = make_classification(n_samples=200, n_features=4, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=0
)
clf = LogisticRegression(max_iter=200)
clf.fit(X_train, y_train)
print("accuracy:", round(accuracy_score(y_test, clf.predict(X_test)), 3))

k-nearest neighbors

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score

X, y = make_classification(n_samples=200, n_features=4, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=0
)
scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)
knn = KNeighborsClassifier(n_neighbors=5)
knn.fit(X_train_s, y_train)
print("accuracy:", round(accuracy_score(y_test, knn.predict(X_test_s)), 3))

Neighbors need scaling: “near” is Euclidean distance. Logistic regression is happier scaled too, but k-NN fails loudly when one column is huge.

Same API

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score

X, y = make_classification(n_samples=200, n_features=4, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=0
)
for name, model in [
    ("logreg", LogisticRegression(max_iter=200)),
    ("knn", KNeighborsClassifier(n_neighbors=5)),
]:
    model.fit(X_train, y_train)
    acc = accuracy_score(y_test, model.predict(X_test))
    print(name, round(acc, 3))

Pitfall

KNeighborsClassifier stores the training set. Keep n_samples in the low hundreds here so predict stays fast.