AI Comparison Methodology

Why we compare AI answers

People increasingly ask AI systems questions about exercise, food, sleep and health. A fluent answer can still omit safety context, rely on weak evidence or sound more certain than the underlying research. FitFat AI comparisons are designed to make those differences visible rather than crown a universal winner.

What every published comparison must disclose

Each experiment records the model and product name, the access tier or mode when relevant, the test date, the exact prompt and whether browsing or other tools were enabled. Models receive the same core question. Any follow-up prompt is reproduced or summarized clearly.

We distinguish the model named by its provider from assumptions about an underlying system. If the interface does not disclose an exact model version, we say so.

How answers are evaluated

We examine whether an answer:

• addresses the actual question;
• distinguishes general information from individual medical advice;
• identifies important risks and exceptions;
• expresses uncertainty appropriately;
• provides sources that exist and support the claim;
• avoids invented statistics or citations;
• offers practical, proportionate next steps;
• corrects itself when challenged with contrary evidence.

Scores, when used, follow criteria published in the article. They represent performance on that prompt and date, not the permanent quality of an entire model.

Fact-checking and limitations

Consequential claims are checked against primary public-health guidance, recognized professional organizations or peer-reviewed research. We link the verification sources separately from sources suggested by the AI.

AI systems change frequently and may produce different answers to the same prompt. A comparison is a documented snapshot, not a reproducible laboratory trial or medical validation. We do not fabricate outputs, silently rewrite an answer to make it stronger, or claim to have tested a model we could not access.

Corrections and provider relationships

Material errors are corrected under our editorial policy. Free access, subscriptions, sponsorships or other relationships with an AI provider must be disclosed in the relevant comparison. Payment does not buy a favorable result.