Error costs choose metrics; metrics do not choose themselves.
10 minutes. Mission: evaluate predictions in business terms.
Start with two errors
A false positive spends a call on someone who declines. A false negative misses someone who would accept. Precision asks how many selected leads accept. Recall asks how many acceptors were found.[1]
Limited call capacity: emphasize precision among the top-ranked leads. Costly missed fraud: emphasize recall, then control investigation load.
Threshold is a business control
The default probability threshold is not sacred. Lowering it usually predicts more positives, increasing recall and often reducing precision. Choose the threshold on validation data, then freeze it before final testing.
Say aloud: “Of predicted positives, how many were right?” Then name its metric. Next: “Of real positives, how many did we find?”
Check retrieval
Win: you can translate false-positive and false-negative costs into a metric and threshold objective.