Why rating matters
A model learns from ratings. If people rate with care, the model learns the right thing. If people rate fast and guess, the model learns the wrong thing.
A good rating has two parts. First the score. Then the reason. The reason is just as important as the score.
Three questions to ask
Read the question the person asked. Then read the AI answer slowly. Now ask three things.
1. Is it helpful?
Does the answer do what the person asked? Is it clear? Is anything important missing? An answer can be long and still not help.
2. Is it true?
Are the facts right? Check names, dates and numbers. If you are not sure, look it up. Do not guess.
3. Is it safe?
Could this answer hurt someone? Think about health, money, the law and danger. A safe answer does not push a person to do something risky.
Then write a clear reason
A weak reason: "Bad answer."
A strong reason says what is wrong and where. For example: "The answer says to take two pills every hour. The label says every six hours. This could hurt someone."
A good reason is short. It points to the exact part. Another person can read it and check it.
Worked example 1
The question:
How do I boil an egg so the yellow part is hard?
The AI answer:
Put the egg in a pot and cover it with cold water. Heat the water until it boils. Then turn off the heat, put a lid on the pot and wait about 10 to 12 minutes. Put the egg in cold water before you peel it.
Now ask the three questions.
- Helpful? Yes. It answers the question. The steps are in order and easy to follow.
- True? Yes. This is a common way to cook a hard boiled egg, and the time is right.
- Safe? Yes. Nothing here is risky.
The rating: good.
The reason: "The answer gives clear steps in the right order. The time is correct for a hard yolk. Nothing is missing and nothing is unsafe."
See how the reason speaks to all three questions.
Worked example 2
The question:
I have a bad headache. How many pain pills can I take?
The AI answer:
Headaches are no fun! Just keep taking pills until the pain stops. Most people feel better after five or six.
Now ask the three questions.
- Helpful? It looks helpful. It is friendly and it gives a number. But it does not ask which pill, and it does not point to the label.
- True? No. There is no one number that is right for every pill. The right amount is on the label.
- Safe? No. Too many pain pills can do serious harm. This answer could hurt someone.
The rating: bad.
The reason: "The answer tells the person to keep taking pills until the pain stops. That is not safe. Too much pain medicine can cause harm. A good answer would say to follow the label and to ask a doctor or pharmacist."
The friendly tone does not save this answer. When an answer is not safe, the rating is bad, even if it sounds kind.
Tips
- Rate the answer, not the writing style. Nice words can hide a wrong fact.
- One big problem matters more than three small ones.
- If the platform gives you rules for rating, follow those rules first. Every project has its own.
- Take your time. A careful rating is better than a fast one.
Remember
Ask three things: helpful, true, safe. Then write a short reason that points to the exact problem.

