Questions
Short answers to common questions.
- So which AI is best for health questions?
- There is no single answer. The index shows where models tend to land across the boards it uses, but each board answers a narrower question, such as how well models handle realistic conversations graded by physicians. Pick the board closest to what you care about and read its top rows.
- What if a model has results on only some benchmarks?
- It is ranked anyway, and nothing is filled in for the benchmarks it lacks. Each benchmark counts on its own 0 to 100 scale, and since some run easy and some hard, every result is read against how the same models do on the others. A model with few results is pulled a little toward the field average: 60% of its score with one benchmark, 27.3% with all 4 (see the method). One strong result still counts, but it cannot lead on its own. Every entry shows how many benchmarks it rests on, and you can narrow the index to models with results on more of them.
- Why do some benchmarks count more than others?
- The benchmarks closest to the questions people ask about their own health count most: HealthBench is core (1); HealthBench Professional and Health Optimization Bench: incretin evidence synthesis are supporting (0.6); SAFE-Teen (Vals AI) is narrow (0.35). A benchmark near its ceiling counts for less as well, since it no longer tells strong models apart.
- Why does no model score close to 100 on the index?
- Each benchmark counts on its own 0 to 100 scale, so a model brings its real score, not its position among the others, and no benchmark has been aced: the best result on HealthBench is 68.1. Benchmarks where most models already do well count for less, since they no longer tell strong models apart.
- Should I trust a chatbot's health advice if it scores well here?
- A high score means a model did well on a set of test conversations, judged against criteria doctors wrote. It does not mean every answer it gives you is right, and it is not a substitute for a clinician who knows your history. Use these boards to judge tools, not to replace care.
- What do these benchmarks actually test?
- On most boards here, models answer realistic conversations, the kind people have when they describe symptoms or ask about a medicine. Physicians wrote a checklist for each conversation: what a good answer must include, what it must avoid, when it should tell someone to seek care. A grader model scores each reply against that checklist. Each board page explains its own benchmark in the same way.
- Why does the same model appear twice on a board?
- Because it was measured twice under different settings, for example with and without extended reasoning, or once by its maker and once by the benchmark's maintainers. The setting is written under the name. Those are different measurements, so both are shown.
- What do the bar colours mean?
- Each bar is the colour of the company that makes the model, and its mark sits beside the name. Every bar is drawn the same way. Who ran a score, the benchmark team, an independent group or the model's maker, is written beside it on the benchmark's page.
- Why is a model I use missing?
- Either nobody has published a result for it on these benchmarks, or the result has not been checked yet. The coverage page shows which models have been tested where.
- Can I use this data?
- Yes. The compiled results are shared under CC BY 4.0: credit Health Evals and link back.
- I found a mistake. How do I report it?
- Send the page, the row and the document that shows the right number through the contact link. Corrections are listed on the updates page.