Data Science Data Dictionary

How it works

A data dictionary explains columns well enough for somebody else to use them. Include the observation unit, measurement unit, source, allowed values and missingness policy. Zero litres can mean an open day with no refills; a missing value can mean the meter failed. Those cases lead to different conclusions and should not be merged just to make a chart easier.

Run the code

Try in playground opens the correct python editor and preloads the complete source. Choose Run to execute; loading does not execute it. The isolated browser runtime does not read personal PC files.

Local review: Python 3.12.14, NumPy 2.5.4, Pandas 3.0.6 and SciPy 1.18.0. Browser review: Pyodide 0.27.7, Python 3.12.7, NumPy 2.0.2, Pandas 2.2.3 and SciPy 1.14.1 in their respective editors. Save the download as data-dictionary.py and run python data-dictionary.py in your separate environment. Apache and MySQL are not needed. Environment setup.

Download the complete source

Example

dictionary = {"litres": {"unit": "L", "minimum": 0, "missing": "meter unavailable"}, "opening_hours": {"unit": "hour", "minimum": 0, "maximum": 12}}
assert dictionary["litres"]["missing"] != "zero demand"
assert dictionary["opening_hours"]["maximum"] == 12
print("Units and missing-value meaning documented")

Expected result

Units and missing-value meaning documented

The dictionary is a human-readable contract represented as a small object. A later validation function must actually enforce its rules; documentation alone does not reject a bad record. Keep the version of the dictionary with the dataset so a changed meter or opening policy does not silently alter the interpretation of old rows.