AI vs Human Attractiveness Ratings: Ranking Is Not Calibration

Correlation can coexist with score bias

A model can preserve the order of many images while shifting scores upward or squeezing them into a narrow band. Reading the correlation alone would miss that difference. For a product promising a human-aligned score, we need error metrics and calibration checks on the actual deployed model and prompt, not a general claim about AI capability.

Panel and question define the target

Different raters, contexts and rating questions can produce different distributions. Calibration must therefore name the target it is calibrating to. A mapping fitted to one convenience sample does not become a population percentile for every user. We would publish the panel definition and uncertainty of the evaluation rather than describe a score as universal.

Our practical product decision

Generic face analysis now uses the six-dimension framework without a synthetic total copied from PSL geometry. The free step offers measured evidence. The optional single-portrait AI report now offers six reference dimension scores and a 100-point total, calculated as their rounded equal-weight mean under our authored rubric. It includes observations, comparisons and a stated evidence status; the numbers have not been calibrated to a human panel. The specialist PSL and golden-ratio tools keep their separately published scales and definitions.

What would justify a numeric perceptual prediction

A dedicated predictor would need independent test images, consent and permissions, a frozen protocol, out-of-sample calibration and a published error analysis. Repeating the same language-model request until a desirable number appears would not meet that standard. A new model or prompt version needs reevaluation.

What we have adopted

Informs clear labeling of our AI reference scores and the need to test calibration on our deployed model, prompts and target panel.

Our public methodology documents the deployed evidence layers and proposed validation protocol. For a concrete example, inspect the complete sample report. The example is illustrative, not an experimental validation result.

Continue reading