Pandas Filling Missing Values

How it works

A missing free-text note can use a display fallback without claiming a recorded note exists. A missing quantity should not usually become zero just because a sum is easier to calculate. Statistical imputation also needs a justified method and a way to distinguish observed from filled values. Preserve a missingness indicator before replacing anything important.

Run the code

Select Try in playground to load the complete editable source. Nothing executes until you choose Run. The browser uses Pyodide 0.27.7, Pandas 2.2.3 and NumPy 2.0.2. Each ordinary execution starts a fresh isolated runtime. The tables are fictional and do not use personal files.

For local work, use a separate virtual environment and run python fill-policy.py. Download the complete source. The reviewed local environment uses Python 3.12.14, Pandas 3.0.6 and NumPy 2.5.4; the browser uses Pyodide 0.27.7 with Pandas 2.2.3 and NumPy 2.0.2. Compatible examples are checked in both. See the environment setup. No Apache or database server is required.

Example

import pandas as pd
import numpy as np

table = pd.DataFrame({"note":pd.Series(["covered", None], dtype="string"), "kg":[12., np.nan]})
result = table.assign(note_missing=table["note"].isna(), note=table["note"].fillna("No note supplied"))
print(result["note"].tolist())
print(result["note_missing"].tolist())
print(result["kg"].isna().tolist())

Expected result

['covered', 'No note supplied']
[False, True]
[False, True]

The note fallback is accompanied by its provenance flag, while the unknown weight stays unknown. That separation prevents interface wording from becoming a fabricated numerical observation. A later export should preserve the distinction when it matters.