Validating Face Analysis: Independent People, Independent Ratings

A new photograph is not always a new person

If the same person appears in development and testing, a model can benefit from identity overlap. Our proposed protocol keeps all photos of a participant in one partition. Multiple photos remain valuable for testing stability, but they must not silently inflate the independence of the evaluation.

Evaluate the complete deployed system

Preprocessing, crop selection, model weights, prompts and calibration all affect a result. We would freeze these before the final test and record every relevant version. An attractive benchmark number from a paper cannot validate a different pipeline, especially when its user images differ from the study’s photographic setting.

Publish several views of performance

Our protocol includes correlation, ranking agreement, absolute error, calibration, repeatability and same-person cross-photo variation. Subgroup comparisons need adequate sample sizes and transparent uncertainty. Rater disagreement is reported separately from model error. We do not infer protected traits from photographs to assign evaluation groups.

Research status versus a marketing claim

Our recruitment and evaluation plans are proposed work. We have not completed an independent human-rating study for the current service. Until that changes, the public description is research-informed photo analysis. Even after validation, a claim must identify the tested model version and target panel rather than promise accurate judgments for every person.

What we have adopted

We adopt the independent-test principle. We do not adopt personality, trustworthiness, health, threat or dominance predictions from appearance.

Our public methodology documents the deployed evidence layers and proposed validation protocol. For a concrete example, inspect the complete sample report. The example is illustrative, not an experimental validation result.

Continue reading