Selecting & filtering

Most pandas work is selection: the right rows, the right columns, in the right order. Learn three tools well — loc, iloc and boolean masks — and almost every other selection is a variation.

The basics

df["sales"]                         # one column
df[["name", "sales"]]               # several columns
df.loc[df["city"] == "Delhi", "sales"]
df.iloc[0:2]                        # first two rows by position
df.query("sales > 100000 and city == 'Delhi'")

loc versus iloc

	loc	iloc
Selects by	Label (index value, column name)	Integer position
End of a slice	Included	Excluded
Boolean mask	Allowed	Not allowed
Typical use	Real work	Peeking at the first/last few rows

df.loc[3:5]        # rows labelled 3, 4 AND 5
df.iloc[3:5]       # rows at positions 3 and 4
df.loc[:, "city":"sales"]      # a span of columns by name
df.iloc[-3:, :2]               # last three rows, first two columns

Always select rows and columns in one loc call: df.loc[mask, "sales"], never df[mask]["sales"].

Boolean masks

delhi = df["city"] == "Delhi"
big = df["sales"] > 100000

df.loc[delhi & big]
df.loc[~delhi, ["name", "sales"]]        # everything except Delhi
df.loc[df["city"].isin(["Delhi", "Pune"])]
df.loc[df["sales"].between(90000, 140000)]
df.loc[df["name"].str.startswith("A")]
df.loc[df["sales"].isna()]

Use &, | and ~ with brackets around each condition. Python's and / or raise ValueError: The truth value of a Series is ambiguous.

query: readable filters

threshold = 100000
top = df.query("sales > @threshold and city in ['Delhi', 'Pune']")

@name pulls in a Python variable. query is easier to read for long conditions and avoids repeating df[...], though column names with spaces need backticks.

Selecting columns by type or pattern

df.select_dtypes("number")
df.select_dtypes(include=["object", "category"])
df.filter(like="sales")            # columns containing 'sales'
df.filter(regex=r"_2025$")

Sorting and picking the top rows

df.sort_values(["city", "sales"], ascending=[True, False])
df.nlargest(3, "sales")            # faster and clearer than sort + head
df.nsmallest(3, "sales")
df.sort_values("sales").head(3)    # same result, more work

A realistic filter

Bills from Pune stores in the festive window, above ₹5,000, paid by UPI:

festive = (
    bills.loc[
        bills["city"].eq("Pune")
        & bills["bill_date"].between("2025-10-01", "2025-11-15")
        & bills["amount"].gt(5000)
        & bills["payment"].eq("UPI"),
        ["bill_id", "bill_date", "amount"],
    ]
    .sort_values("amount", ascending=False)
)

The .eq / .gt / .between methods keep long conditions readable and avoid bracket mistakes.

Common mistakes

• Chained indexing (df[mask]["col"] = 0), which may write to a temporary copy.
• Expecting df.loc[0:5] to return five rows; label slices include the end.
• Using == against None or NaN; use .isna().
• Filtering on a text column without cleaning it first, so "delhi" and "Delhi " are missed.