pandas uses NaN (float) for missing numbers, and often NaT for missing timestamps. Missing values propagate: NaN + 1 is still NaN. Count them before you drop or fill.

Goal

Detect, drop, and fill missing cells, and know when filling is lying vs when dropping is bias.

Find missing

df = pd.DataFrame(
    {
        "name": ["Ada", "Alan", "Grace", "Linus", "Nia"],
        "score": [98, np.nan, 95, 88, np.nan],
        "team": ["A", "B", None, "B", "A"],
        "attempts": [1, 2, 1, np.nan, 3],
    }
)
print(df)
print()
print(df.isna())
print()
print(df.isna().sum())
print()
print("rows with any NA:")
print(df[df.isna().any(axis=1)])

notna() is the inverse. None, np.nan, and pandas NA all count as missing here.

Drop

df = pd.DataFrame(
    {
        "name": ["Ada", "Alan", "Grace", "Linus", "Nia"],
        "score": [98, np.nan, 95, 88, np.nan],
        "team": ["A", "B", None, "B", "A"],
        "attempts": [1, 2, 1, np.nan, 3],
    }
)
print("drop rows with any NA")
print(df.dropna())
print()
print("drop rows missing score")
print(df.dropna(subset=["score"]))
print()
print("drop columns that are all NA (none here)")
print(df.dropna(axis=1, how="all"))

Fill

df = pd.DataFrame(
    {
        "name": ["Ada", "Alan", "Grace", "Linus", "Nia"],
        "score": [98, np.nan, 95, 88, np.nan],
        "team": ["A", "B", None, "B", "A"],
        "attempts": [1, 2, 1, np.nan, 3],
    }
)
print(df.fillna({"score": df["score"].median(), "team": "unknown", "attempts": 0}))
print()
print("forward fill team")
print(df.assign(team=df["team"].ffill()))

ffill / bfill copy the previous / next valid value. They are for ordered series (time), not random tables.

Interpolate (ordered numeric)

s = pd.Series([10.0, np.nan, np.nan, 40.0])
print(s.interpolate())

Arithmetic with NA

df = pd.DataFrame({"score": [98.0, np.nan, 95.0, 88.0]})
print(df["score"] + 1)
print()
print("mean skips NA by default:", df["score"].mean())
print("mean if you force NA to count:", df["score"].mean(skipna=False))

Pitfall

df["score"].fillna(0) can hide that exams were never taken. For totals, filling with 0 may be right. For averages, dropping or using skipna is usually righter. Write down which you chose.