← All lessons

NaN and infinity

isnan, nanmean, nansum, nan_to_num, and why nan != nan.

nan is a float. It is not equal to itself. Reductions like mean become nan if any cell is missing unless you use nanmean.

Goal

Detect missing cells, fill them, and use nanmean / nan_to_num on a gappy rainfall grid.

nan != nan

x = np.nan
print("x == x:", x == x)
print("np.isnan:", np.isnan(x))

Never write arr == np.nan. Use np.isnan(arr).

Contagious mean

# Rows: Nairobi, Mombasa, Kisumu  ·  cols: Jan–Apr
rain = np.array(
    [
        [50, 40, 80, np.nan],
        [20, 15, np.nan, 90],
        [70, 80, 120, 180],
    ],
    dtype=float,
)
print("mean:", np.mean(rain))
print("nanmean:", np.nanmean(rain))
print("nansum:", np.nansum(rain))
print("nan count:", np.isnan(rain).sum())

Fill with the city mean

rain = np.array(
    [
        [50, 40, 80, np.nan],
        [20, 15, np.nan, 90],
        [70, 80, 120, 180],
    ],
    dtype=float,
)
city_mean = np.nanmean(rain, axis=1, keepdims=True)
filled = np.where(np.isnan(rain), city_mean, rain)
print("city means:")
print(city_mean)
print("filled:")
print(filled)

nan_to_num and inf

vals = np.array([50, np.nan, np.inf, -np.inf], dtype=float)
print(np.nan_to_num(vals, nan=0.0, posinf=999, neginf=0))
print(np.isfinite(vals))
Pitfall

astype(int) after a nan is undefined. Stay in float until missing cells are filled.