Human in the Loop for Generative AI Content

Human in the loop for generative AI content means a named person reads AI-written text before it reaches the people it is for, checks it against a short list written for that kind of content, and signs it off, and the sign-off is recorded with what was checked. How much review a piece gets depends on who will read it: anything customers or the public will see gets a full read by someone who can say no; internal notes can get less. Where the law asks for a label, as the EU AI Act now does for some content, the review and the label are planned together.

This post is about words: copy, documentation, replies and comments. For reviewing agent work in general, including code and data changes, see verifying AI-generated work. For twelve worked cases across different kinds of work, see human-in-the-loop AI examples.

Why content needs its own loop

Generated text fails differently from generated code. Code that is wrong usually breaks a test. Text that is wrong reads exactly like text that is right, and nothing breaks until a reader acts on it. OWASP’s entry on misinformation in LLM applications↗ (opens in a new tab) names hallucination, content that seems accurate but is fabricated, as a major cause, and cites an airline whose chatbot gave travellers wrong information and which was then successfully sued. Its mitigation is human oversight and fact-checking, with reviewers trained not to over-rely on the output.

Content also travels. A reply is forwarded, a help page is bookmarked, a product claim is quoted. A correction never reaches everyone who read the first version, which is why the check belongs before publishing, not after.

What to check, by kind of content

Marketing copy

• Every claim about the product: does it do this today, on the plan the page is selling?
• Every number, date and name, against its source. Delete any statistic nobody can source.
• Comparisons with other products: fair, current and checkable, or cut.
• Reviews and testimonials: real people, real words. The US Federal Trade Commission’s final rule on fake reviews↗ (opens in a new tab) bans reviews and testimonials that misrepresent that they are by someone who does not exist, such as AI-generated fake reviews.
• Voice: read it aloud once. Fluent but generic is a sign nobody edited it.

Documentation and help pages

• Run every step, in the current version, as a new user would. A step that no longer exists is the most common fault.
• Copy and run every command and code sample exactly as printed.
• Check each screen name, button label and setting against the product itself, not against an older page.
• Say what the page does not cover, rather than letting the model fill the gap.

Support replies

• Does it promise anything the policy does not allow: a refund, a date, a feature?
• Is the stated cause true, and confirmed on this customer’s account?
• Does it contain anyone’s personal data that should not be there, including another customer’s?
• Would you be comfortable seeing it quoted back to you? Tone matters most when the customer is upset.

Code comments, commit messages and release notes

• Does the comment describe what the code does now, not what it did before the change or what the model assumed?
• No invented reasons. “Done for performance” with no measurement behind it misleads the next reader.
• Release notes list only what actually shipped. Match each line to a finished task.

Who signs off

One named person per piece, not “the team”. Usually that is the person who asked for it, because they know what it is for. Add a second reader where the first cannot judge: a product owner for claims about features, an engineer for technical docs, and, for anything regulated or contractual, whoever is responsible for that in your organisation. The model drafted it; the person who signs it off answers for it. If nobody is prepared to put their name to a piece, it is not ready.

• Public or customer-facing: full review every time, by someone with authority to stop it.
• Internal and widely read, such as a company announcement or a process page: reviewed by the person who asked for it.
• Internal drafts and notes for yourself: no formal review, but label them as drafts.
• High-volume routine replies built from approved templates: sample a fixed number each week, and review in full anything outside the template.

How to record it

A sign-off nobody can find did not happen, as far as the next person is concerned. Keep one short record per piece, next to the work, not in someone’s inbox.

Piece:       Help page "Export your board to CSV"
Drafted by:  AI assistant, from FET-022 and the export code
Reviewed by: product owner
Checked:     every step run on the live app; both code samples run;
             screenshots retaken
Not checked: behaviour on boards over 10,000 tasks
Label:       none needed (help page, human reviewed)
Published:   /help/export, 27 Sep

On a fenbs board, each piece can be a task, and its Testing section holds the verdict: Tested, Partly tested, Failed, or Needs owner check when only a particular person can confirm it, with notes saying what was checked and what was not. An assistant that drafts the page sets the status honestly and leaves the move to Completed to the reviewer. Every change is recorded in History with who made it. A standing rule, such as “every AI-drafted help page is read by the product owner”, can be recorded on the Decisions page, where a decision is made by people and never by an AI assistant.

Disclosure and labelling

In the EU, Article 50 of the AI Act↗ (opens in a new tab) has applied since 2 August 2026. Providers of generative systems must mark their outputs in a machine-readable way so they can be detected as AI-generated. Deployers, the organisations using those systems, must disclose deepfakes, meaning image, audio or video that resembles real people, places or events and would falsely appear authentic. And AI-generated or manipulated text published to inform the public on matters of public interest must be disclosed, unless it has been through human review or editorial control and a person or organisation holds editorial responsibility for it.

That last exception is the human loop written into law: a named editor who reviewed the piece and answers for it changes what you have to disclose. It is worth keeping the review record above for that reason alone. The Commission has also published a code of practice on marking and labelling AI-generated content↗ (opens in a new tab), a voluntary tool for showing compliance, alongside guidelines on the scope of the transparency rules. Whether a particular page counts as informing the public on matters of public interest is a question for those guidelines and, where it matters, a lawyer. Human oversight under the EU AI Act summarises the rest of the Act and its dates.

Outside the law, being open about how content was made is a matter of trust. Google’s guidance on generative AI content↗ (opens in a new tab) asks for a focus on accuracy, quality and relevance, warns that generating many pages without adding value for users may violate its spam policy, and suggests telling readers how automatically generated content was created where that makes sense for your audience.

Where the loop quietly breaks

• The reviewer reads the draft, then someone edits it after sign-off. Review the version that ships.
• The reviewer checks tone and spelling but not facts, because the text sounds sure of itself.
• Approval becomes a click. If a reviewer passes forty pieces a day and never sends one back, the loop is on paper only.
• “The AI wrote it” is used to explain an error. The person who signed it off owns it, which is the point of signing.