Study Overview
This research partnership with Quill and the Chan Zuckerberg Initiative (CZI) develops standardized evaluation methods to compare AI-generated feedback against feedback from experienced educators. We're building benchmark datasets and evaluation protocols to measure how well AI systems perform in providing writing instruction feedback.
Product Description
Quill is an AI-powered writing platform that provides real-time feedback to students on their writing. Students will be completing three activities where they read an article and then complete a series of three sentence-level writing prompts. In each prompt, students build a sentence using evidence from the text.
Participant Responsibilities/ Research Activities
As annotators, you will be asked to review and annotate student responses by writing 2–3 sentences of feedback on each. In this role, you’ll bring your expertise as an ELA teacher to help build a high-quality dataset that reflects authentic classroom feedback practices
(~30 hours per teacher per evaluation cycle, $50/hr).
Annotators are NOT required to use Quill with their students or do work within school hours.
Assessment Required
None
Agreements Required
Individual Contractor Agreements
Ongoing Use/Cost
Quill.org is free for all students and teachers. Partners in the study will also receive free access to Quill Premium, which provides teachers with additional data reporting and professional development.
Benefits
Contribute to AI education research - Help shape the future of AI writing tools by participating in research that improves feedback quality
Access to open-source protocols - Schools and educators will have access to the evaluation methods and datasets developed through this partnership, enabling broader adoption of AI writing tools
Compensation
Teacher annotators: $50/hour (~30 hours per evaluation cycle). It is ideal if you can contribute to all 3 cycles but not required.
Timeline
| October - December 2025 |
Recruitment
|
||||
| January 2026 | Onboarding |
||||
| February - March 2026 |
Cycle 1| April - June 2026 |
Cycle 2 | July - September 2026 |
Cycle 3 | |