AI for Writing Exam Questions: Prompts and Quality Checks

AI drafts exam questions quickly, but a generic prompt produces generic questions: too easy, too vague, or testing the wrong thing. Specifying the learning objective, the question type, and the cognitive level produces questions that are actually usable and whose answer keys you can verify.

Tools like fenbs let you save the prompts that produced good questions and the decisions about which assessment rules apply to your course, so the next set of questions starts from a tested process rather than a rewritten one.

What good exam questions require

A good exam question is aligned to a specific learning objective, asks for evidence of understanding rather than recall of phrasing, and has an answer key you can justify. AI can help with the first draft of each of those, but it needs the learning objective in the prompt or it will guess.

The most common failure mode is asking for questions about a topic without specifying the cognitive level. A question that asks "what is photosynthesis?" tests recall. A question that asks "explain why a plant grown in a low-CO2 environment would produce less glucose at the same light level" tests understanding. The difference is in the prompt, not in the AI.

A well-phrased learning objective makes the cognitive level choice easier. "Students will identify the parts of photosynthesis" suggests recall. "Students will explain the effect of variable CO2 on glucose production" suggests understanding or application. Paste the learning objective directly into the prompt and the appropriate cognitive level often follows from it without requiring a separate instruction.

Aligning questions to learning objectives

Start every question-writing session by stating the learning objective the question is meant to assess. The AI cannot align a question to an objective it does not know.

Write [number] [question type] questions that assess the following learning objective:
[paste the learning objective]

Cognitive level: [recall / understanding / application / analysis]
Context to use: [topic area or source material]
For each question, provide the correct answer and a brief explanation of why it is correct.

The "cognitive level" field is the most useful constraint. It tells the AI whether to ask for a definition (recall), an explanation (understanding), a worked example (application), or a comparison of two cases (analysis).

Prompts for different question types

Different question types need slightly different prompt instructions:

• Multiple choice: "Write four options. The correct answer must be unambiguously right. The three distractors must reflect common misconceptions, not trick phrasing."
• Short answer: "The question must be answerable in two to four sentences. Include the key points the answer must contain."
• Essay or long answer: "State the word limit and the criteria for a strong answer. Do not use question phrasing that would be satisfied by a list of definitions."
• Scenario-based: "Give a brief, realistic scenario first, then ask a question that requires applying the concept to it."

For multiple choice, the distractors are where most of the drafting time goes. Ask the AI to explain why each distractor is a plausible wrong answer: that check reveals whether the distractor reflects a real misconception or just a vaguely different phrasing.

Verifying the answer key

Always verify the answer key before using a question. AI-generated answers are often correct but can be incomplete, slightly off, or phrased in a way that makes a borderline student answer look wrong when it is actually right.

1. Read each question and answer together and check that the answer follows from the question without additional assumptions.
2. For multiple-choice distractors, check that no distractor is defensible as correct under a reasonable reading.
3. For open-ended questions, check that the marking criteria are specific enough to distinguish a strong answer from an adequate one.

A useful follow-up prompt: "For the question above, what is a common incorrect answer a student might give, and what misconception does it reveal?" This surfaces the diagnostic value of the question beyond whether it can be marked.

If your institution uses FERPA or other student privacy rules, those govern how student responses can be shared with any tool, including an AI. Check your institution's policy before pasting any student-identifiable content into a prompt.

How to do this with fenbs

1. Create a free account and add a project for the course or assessment.
2. Add the question-writing prompts to Skills and prompts in fenbs: one for each question type you use regularly.
3. Record assessment rules and decisions, such as which cognitive levels each assessment targets, in Decisions and rules.
4. Connect your AI app over MCP from Integrations so it reads the current prompts and assessment decisions before each question-writing session.

Where to be careful

AI-generated questions work best as a starting point, not a finished product. They require review by someone who knows the course material and the students' likely level. The AI does not know which topics your class covered in depth and which were mentioned briefly.

For writing and essay questions, AI can produce plausible rubrics, but the decision about what counts as a strong versus adequate answer is a professional judgment. Use the AI to generate candidates; make the judgment yourself.

For AI tools in education more broadly, how to use ai at work covers the broader question of where AI assistance adds value and where human judgment is required.