← All lessons

Group by

Split-apply-combine with agg, transform, filter, size, and named aggregations.

groupby splits the table on key values, applies a reduction (or transform) to each piece, then combines the results. This is how you get “revenue by city” without a pivot table UI.

Goal

Aggregate with named tuples, transform to align back to rows, and count with size vs count.

Split, apply, combine

df = pd.DataFrame(
    {
        "city": ["Nairobi", "Nairobi", "Mombasa", "Mombasa", "Kisumu", "Nairobi"],
        "product": ["A", "B", "A", "A", "B", "A"],
        "units": [12, 7, 9, 4, 11, 5],
        "price": [10.5, 22.0, 10.5, 10.5, 22.0, 10.5],
    }
)
df["revenue"] = df["units"] * df["price"]
print(df.groupby("city")["revenue"].sum())
print()
print(df.groupby("city")["revenue"].sum().reset_index())

Without reset_index(), the group key is the index.

Several keys and named aggregations

df = pd.DataFrame(
    {
        "city": ["Nairobi", "Nairobi", "Mombasa", "Mombasa", "Kisumu", "Nairobi"],
        "product": ["A", "B", "A", "A", "B", "A"],
        "units": [12, 7, 9, 4, 11, 5],
        "price": [10.5, 22.0, 10.5, 10.5, 22.0, 10.5],
    }
)
df["revenue"] = df["units"] * df["price"]
print(
    df.groupby(["city", "product"], as_index=False).agg(
        units=("units", "sum"),
        revenue=("revenue", "sum"),
        orders=("units", "count"),
        avg_price=("price", "mean"),
    )
)

The pattern is new_name=("column", "function"). Functions can be "sum", "mean", "median", "min", "max", "count", "nunique", "std", "first", "last".

size vs count

df = pd.DataFrame(
    {
        "city": ["Nairobi", "Nairobi", "Mombasa", "Kisumu"],
        "units": [12, np.nan, 9, 11],
    }
)
print("rows per city (size)")
print(df.groupby("city").size())
print()
print("non-NA units per city (count)")
print(df.groupby("city")["units"].count())

size counts rows. count skips NA in that column.

transform — same length as the original

df = pd.DataFrame(
    {
        "city": ["Nairobi", "Nairobi", "Mombasa", "Mombasa", "Kisumu"],
        "product": ["A", "B", "A", "B", "A"],
        "revenue": [126.0, 154.0, 94.5, 132.0, 31.5],
    }
)
df["city_total"] = df.groupby("city")["revenue"].transform("sum")
df["share"] = df["revenue"] / df["city_total"]
print(df)

Use transform when you need a group statistic on every row (shares, z-scores, fill-with-group-median).

filter groups

Keep groups that pass a test.

df = pd.DataFrame(
    {
        "city": ["Nairobi", "Nairobi", "Mombasa", "Kisumu"],
        "revenue": [126.0, 154.0, 94.5, 31.5],
    }
)
print(df.groupby("city").filter(lambda g: g["revenue"].sum() >= 150))

Grouped top-n

df = pd.DataFrame(
    {
        "city": ["Nairobi", "Nairobi", "Mombasa", "Mombasa", "Kisumu"],
        "product": ["A", "B", "A", "B", "A"],
        "revenue": [126.0, 154.0, 94.5, 132.0, 31.5],
    }
)
ordered = df.sort_values(["city", "revenue"], ascending=[True, False])
print(ordered.groupby("city").head(1))
Tip

as_index=False keeps group keys as columns — friendlier for a later merge or to_csv.