Fix pandas SettingWithCopyWarning silently corrupting a derived column
The problem
After filtering a DataFrame and computing a column on the result, the original frame showed NaN where the derived values should be — while pandas printed:
SettingWithCopyWarning: A value is trying to be set on a copy of a DataFrame from a slice of a DataFrame.
No exception, downstream numbers quietly wrong.
What didn't work
pd.set_option('mode.chained_assignment', None)— hides the warning; the missing write stays missing.- Assuming it "sometimes works" — chained indexing (
df[mask]['col'] = x) lands on a temporary object, and whether pandas optimizes it into a view or a copy is not guaranteed, so the same line succeeds on one frame and silently no-ops on another. - Assigning via
.values— hides the index alignment and produces rows matched by position, corrupting data when the filtered index isn't a clean range.
The fix
Write through .loc on the parent frame in a single call:
import pandas as pd
df = pd.read_csv("orders.csv")
mask = df["revenue"] > 0
df.loc[mask, "margin"] = df.loc[mask, "profit"] / df.loc[mask, "revenue"]
Or, when you genuinely want a working copy, sever the link explicitly:
subset = df[df["region"] == "EMEA"].copy()
subset["margin"] = subset["profit"] / subset["revenue"]
df.loc[subset.index, "margin"] = subset["margin"] # write back by index, not by slice
Why it works
The warning fires because pandas cannot tell whether the intermediate slice is a view or a copy; a single .loc write on the parent needs no such guess, and an explicit .copy() removes the ambiguity by contract instead of by optimization luck.