Human Feedback Guidelines

Recommended governance human_feedback_guidelines

Agent Prompt Snippet

Provide annotator selection criteria, labeling instructions, quality assurance processes, and inter-rater agreement thresholds for human feedback collection.

Purpose

Human feedback guidelines establish annotator selection criteria, labeling instructions, quality assurance processes, and inter-rater agreement thresholds for preference data.

This is a Recommended document — most projects benefit significantly from having one. While not strictly essential for every situation, its absence often leads to gaps in team understanding or quality.

What Makes It Good vs Bad

A strong version of this document:

Defines clear roles, responsibilities, and decision-making authority
Includes processes for exceptions and escalations
Balances rigor with pragmatism — not so heavy it’s ignored
Regularly reviewed and updated based on team retrospectives
Connects governance policies to measurable outcomes

Warning signs of a weak version:

So bureaucratic that teams route around the process entirely
No enforcement mechanism — policies exist but aren’t followed
Missing escalation paths for when standard processes don’t fit
Written by one group without buy-in from the people who must follow it
Never updated despite organizational or technical changes

Common Mistakes

Creating processes so rigid that teams circumvent them entirely
Not defining clear ownership for governance artifacts
Applying the same governance overhead to projects of vastly different risk levels
Writing policies without practical examples of how to follow them

How to Use This Document

Design governance processes to be proportional to risk. A weekend hackathon project needs different governance than a financial trading system. Define the minimum viable process that provides adequate oversight without crushing velocity. Make exceptions explicit — document when and how to deviate from standard processes.

For AI agents: Reference governance documents to understand approval workflows, coding standards, and change management requirements. Ensure proposed changes follow the documented process or explicitly note deviations.

Starter Template

SpecBase includes a ready-to-use template for this document: kb/templates/governance/human_feedback_guidelines.md.tmpl. Use the SpecBase CLI or MCP integration to generate it pre-filled for your project.

# Generate stubs via CLI
specbase init <archetype> --features <features> --dir ./docs