bhylu.com

Advertisements

Data Science Variance

State whether your calculation describes the observed set or estimates a population spread.

How it works

Variance summarises squared deviations from the mean. It has squared measurement units, so litres produce square litres. NumPy’s default denominator is n; setting ddof=1 uses n-1, commonly used for sample variance under its assumptions. The denominator choice should follow the interpretation of the data, not which result is more convenient. At least two observations are needed for the latter calculation.

Run the code

Try in playground opens the correct numpy editor and preloads the complete source. Choose Run to execute; loading does not execute it. The isolated browser runtime does not read personal PC files.

Local review: Python 3.12.14, NumPy 2.5.4, Pandas 3.0.6 and SciPy 1.18.0. Browser review: Pyodide 0.27.7, Python 3.12.7, NumPy 2.0.2, Pandas 2.2.3 and SciPy 1.14.1 in their respective editors. Save the download as variance.py and run python variance.py in your separate environment. Apache and MySQL are not needed. Environment setup.

Download the complete source

Example

variance.py

import numpy as np
x = np.array([10., 12., 14.])
population = x.var(ddof=0)
sample = x.var(ddof=1)
assert np.isclose(population, 8/3)
assert sample == 4.
print(f"Observed-set variance {population:.3f}; sample variance {sample:.3f}")

Expected result

Observed-set variance 2.667; sample variance 4.000

The squared deviations are four, zero and four. Dividing their sum by three describes this complete set; dividing by two supplies the usual sample estimate under independent sampling assumptions. A software default cannot decide whether your days constitute an entire target population or an incomplete sample.

Technical references: NumPy, Pandas and SciPy. For evaluation design see the scikit-learn guide to leakage; these examples do not require scikit-learn.