For companies: explore licensing operational data for AI. Check eligibility

Jobs › School › Lesson 2

Lesson 2: How to rate an AI answer

Helpful, true, safe, and a clear reason for your rating. About 5 minutes. Read the lesson, then answer 3 questions. Get all 3 right to earn the Lesson 2 badge.

Big T is your teacher.

Big T: “A rating helps most when it comes with a reason. Today you learn to give both. Take your time with each answer.”

Why rating matters

A model learns from ratings. If people rate with care, the model learns the right thing. If people rate fast and guess, the model learns the wrong thing.

A good rating has two parts. First the score. Then the reason. The reason is just as important as the score.

Three questions to ask

Read the question the person asked. Then read the AI answer slowly. Now ask three things.

1. Is it helpful?

Does the answer do what the person asked? Is it clear? Is anything important missing? An answer can be long and still not help.

2. Is it true?

Are the facts right? Check names, dates and numbers. If you are not sure, look it up. Do not guess.

3. Is it safe?

Could this answer hurt someone? Think about health, money, the law and danger. A safe answer does not push a person to do something risky.

Then write a clear reason

A weak reason: "Bad answer."

A strong reason says what is wrong and where. For example: "The answer says to take two pills every hour. The label says every six hours. This could hurt someone."

A good reason is short. It points to the exact part. Another person can read it and check it.

Worked example 1

The question:

How do I boil an egg so the yellow part is hard?

The AI answer:

Put the egg in a pot and cover it with cold water. Heat the water until it boils. Then turn off the heat, put a lid on the pot and wait about 10 to 12 minutes. Put the egg in cold water before you peel it.

Now ask the three questions.

  • Helpful? Yes. It answers the question. The steps are in order and easy to follow.
  • True? Yes. This is a common way to cook a hard boiled egg, and the time is right.
  • Safe? Yes. Nothing here is risky.

The rating: good.

The reason: "The answer gives clear steps in the right order. The time is correct for a hard yolk. Nothing is missing and nothing is unsafe."

See how the reason speaks to all three questions.

Worked example 2

The question:

I have a bad headache. How many pain pills can I take?

The AI answer:

Headaches are no fun! Just keep taking pills until the pain stops. Most people feel better after five or six.

Now ask the three questions.

  • Helpful? It looks helpful. It is friendly and it gives a number. But it does not ask which pill, and it does not point to the label.
  • True? No. There is no one number that is right for every pill. The right amount is on the label.
  • Safe? No. Too many pain pills can do serious harm. This answer could hurt someone.

The rating: bad.

The reason: "The answer tells the person to keep taking pills until the pain stops. That is not safe. Too much pain medicine can cause harm. A good answer would say to follow the label and to ask a doctor or pharmacist."

The friendly tone does not save this answer. When an answer is not safe, the rating is bad, even if it sounds kind.

Tips

  • Rate the answer, not the writing style. Nice words can hide a wrong fact.
  • One big problem matters more than three small ones.
  • If the platform gives you rules for rating, follow those rules first. Every project has its own.
  • Take your time. A careful rating is better than a fast one.

Remember

Ask three things: helpful, true, safe. Then write a short reason that points to the exact problem.

This lesson teaches a general skill. Every platform has its own rules and its own steps. Always read theirs.

Try it yourself

Tap an answer. You see at once if it is right, and why. Get all 3 right to finish the lesson. A miss costs nothing: tap Try again. Each question right on the first try earns a star.

  1. Question 1 of 3. Which three things do you check when you rate an AI answer?

  2. Question 2 of 3. Which reason is the best one?

  3. Question 3 of 3. An AI answer is very friendly, but its advice could hurt someone. What is the rating?

Next: Lesson 3 →

More: The school · The Skills Check · Free resume help