Score (SN44)’s Smart mAP as a Fairer Standard for Object Detection

Object detection models get ranked using a single number, mAP, and that number has always rested on one fixed rule for deciding whether a prediction counts as correct.

That rule just changed with Score (SN44)’s introduction of Smart mAP, a new approach that adjusts the threshold based on how large the object is.

This model corrects a bias that has quietly skewed model rankings for years, favoring some object sizes over others regardless of true detection quality.

Here’s what was broken about the old standard, and how the new one fixes it.

Where the Old Standard Broke Down

Mean Average Precision (mAP) works by checking how well a model’s predicted bounding box overlaps with the true location of an object, a measure called Intersection over Union (IoU).

Until now, a single IoU threshold, usually 0.50, decided whether any prediction counted as correct regardless of what was being detected.

That fixed threshold created an uneven playing field:

1. Small objects got punished for tiny errors: A localization mistake of just a few pixels can blow past a 0.50 threshold on a small object, even when the model’s overall detection was accurate.

2. Large objects got a pass on sloppy boxes: The same 0.50 threshold is far easier to clear on a big object, letting genuinely imprecise localization still count as correct.

The result skewed model comparisons, with rankings ending up reflecting how well a model handled the threshold’s blind spots almost as much as how well it actually detected objects.

How Smart mAP Fixes It

Smart mAP scales the IoU threshold to the size of the ground-truth object instead of applying one number to everything:

1. Objects under 1% of the image get a threshold that scales down from 0.50 to as low as 0.30, giving small objects room for the kind of minor localization variance that shouldn’t disqualify an otherwise correct detection.

2. Objects between 1% and 5% of the image keep the standard 0.50 threshold, since this is the size range the original metric was built around.

3. Objects over 5% of the image get a threshold that scales up to as high as 0.70, holding large objects to the tighter precision their size makes achievable.

4. Extreme cases get hard limits. Objects smaller than 0.05% or larger than 25% of the image are capped at 0.30 and 0.70 respectively, so the scaling never runs away in either direction.

A Fairer Number, Not Just A New One

The point of Smart mAP is to make the benchmark more honest about what it’s measuring. Grading small and large objects against the same fixed bar has never reflected how localization difficulty scales with object size in the first place.

Smart mAP corrects that imbalance without changing what mAP fundamentally sets out to measure from the start.

What comes out the other end is a ranking that tracks real detection ability more closely, giving anyone comparing models a signal they can trust.