AI Exercises
Work through an evaluation mistake before reading the result.
How it works
A model reports impressive accuracy because almost every case has the same label. Calculate both its accuracy and positive-class recall. Then explain why a reviewer should ask for counts, a baseline and a suitable hold-out design before calling it useful. The worked calculation deliberately contains a strong-looking aggregate and a complete failure on the positive class.
Run the code
Try in playground loads this complete standard-library Python example without executing it. Choose Run when ready. Local review uses Python 3.12.14; the existing browser editor uses Pyodide 0.27.7. These small fictional examples need no account, external data, model-provider key or database service.
Example
actual = [0]*19+[1]
predicted = [0]*20
correct = sum(a==p for a,p in zip(actual,predicted,strict=True))
positives = sum(actual)
hits = sum(a==1 and p==1 for a,p in zip(actual,predicted,strict=True))
print(f"accuracy: {correct/len(actual):.2f}")
print(f"positive recall: {hits/positives:.2f}")
Expected result
accuracy: 0.95 positive recall: 0.00
The classifier gets nineteen cases right by predicting zero for all of them. It misses the only positive. Try adding predicted positives and track both false positives and recall. Do not simply rebalance the test set to make a preferred metric look better.