Greenhouse ·
full-timeProctor Quality And Learning Manager, Scaling Operations
Duolingo · New York, NY; Pittsburgh, PA
Duolingo is a language‑learning platform that also administers the Duolingo English Test, a high‑stakes assessment used by institutions worldwide. In the Proctor Quality and Learning Manager role within Scaling Operations, you will own the end‑to‑end quality standard and learning system for the global proctor network. Day‑to‑day responsibilities include maintaining rubrics and decision rules that define accurate proctoring, leading calibration and adjudication cycles, designing and delivering ongoing training programs, and measuring their impact through audits and performance trends. You will guide Training Specialists and adjudicators, keep runbooks current, and evolve quality controls as new security threats, technology changes, or workflow updates emerge. The role requires close partnership with Psychometrics, Security, Operations, Product, and Engineering teams to ensure both human and AI‑assisted workflows meet rigorous accuracy standards before and after deployment. Ideal candidates have five or more years of experience in assessment development, scoring/rater quality, certification, or quality assurance, with a background in aligning reviewers through calibration programs and designing adult‑learning initiatives that include onboarding, certification, remediation, and effectiveness measurement. Strong attention to detail, judgment in ambiguous cases, and analytical fluency to interpret quality data are essential.
Our mission at Duolingo is to develop the best education in the world and make it universally available. It’s a big mission, and that’s where you come in! At Duolingo, you’ll join a team that cares about educating our users, experimenting with big ideas, making fact-based decisions, and finding innovative solutions to complex problems. You’ll have limitless learning opportunities and daily collaborations with world-class minds — while doing work that’s both meaningful and fun. Join our life-changing mission to develop education for our half a billion (and growing!) learners around the world. Read our blog to learn more. About the role... The Duolingo English Test (DET) is a high-stakes assessment used by institutions around the world. Its integrity depends on proctors making accurate, consistent, and defensible decisions — even when sessions present ambiguous evidence or new forms of cheating. As the Proctor Quality and Learning Manager (L3) in Scaling Operations, you will own the end-to-end quality standard and learning system that keep our global proctor network aligned and effective as security threats, technology, and workflows evolve. You’ll define what “correct” looks like, run our calibration and adjudication program, and build ongoing training that measurably improves decision quality. You will partner with Psychometrics, Security, Operations, Product, and Engineering to ensure our human and AI-assisted workflows meet rigorous accuracy standards before and after deployment. This is an individual-contributor role within DET Proctoring Operations, with functional leadership over Training Specialists and adjudicators executing the program. 🧠 You will... Own the DET proctoring quality standard — maintain rubrics, decision rules, examples, and guidance that define accurate decisions across routine and difficult edge cases. Lead calibration and adjudication — operate recurring Proctor Certification Test cycles, resolve disputed decisions, identify patterns of disagreement, and ensure outcomes are consistent and auditable. Build and run an ongoing learning program — publish a forward-looking training calendar, deliver monthly interventions, and launch targeted training in response to quality signals, threats, or workflow changes. Measure effectiveness — use calibration results, audits, and performance trends to identify learning needs, evaluate training impact, and decide when to clarify standards, retrain, or update guidance. Guide specialist execution — set expectations for Training Specialists and adjudicators, assign deliverables, review outputs, and operate onboarding and certification that ramp new specialists quickly. Maintain durable sources of truth — keep calibration, adjudication, and training runbooks current so critical processes are documented and auditable. Evolve quality alongside proctoring — translate new threats and workflow changes into clear standards, training, and quality controls in partnership with Psychometrics, Security, Operations, Product, and Engineering. Represent the proctoring craft — bring frontline proctor evidence into product experiments and workflow decisions; assess whether launched changes preserve decision quality. Connect human and technology-assisted quality — establish benchmarks for AI-assisted workflows and validate automated capabilities against real sessions before deployment. ✅ You have... 5+ years in assessment development, scoring/rater quality, certification, quality assurance, or a field involving complex, consequential judgments; experience aligning reviewers to a shared standard through calibration programs. Adult-learning program experience spanning onboarding, certification, ongoing training, remediation, and effectiveness measurement (beyond one-off materials). Exceptional attention to detail and strong judgment in ambiguous cases; ability to produce clear, defensible decisions and guidance. Analytical fluency to interpret quality data, disagreement patterns, and performance trends and translate them into practical interventions. Clear written and verbal communication; ability to turn complex standards into precise, usable guidance and explain difficult calls. Cross-functional leadership without formal authority; experience directing contractors or partners toward clear accountability and outcomes. Demonstrated skill in quality standards and rubric design, building living systems with decision logs, change histories, and controls for auditability. Experience defining quality benchmarks and validation methods for human–technology workflows, including AI-assisted processes. Adaptability — comfort revising standards and programs as risks, products, and technology change. ⭐ Exceptional candidates will have... Experience in language assessment, test security, remote proctoring, trust and safety, or fraud prevention in high-integrity environments. Experience maintaining quality across large, distributed examiner/rater/reviewer/contractor populations. Fami
| Role | Proctor Quality And Learning Manager, Scaling Operations |
|---|---|
| Company | Duolingo |
| Location | New York, NY; Pittsburgh, PA |
| Compensation | Not disclosed |
| Posted | 2026-09-30 |
| Deadline | Rolling |
Typical process for this type of role
A general guide — the exact steps for this specific listing may vary; check the original posting for details.
- 1ApplicationSubmit your resume through the apply link.
- 2ScreeningRecruiter reviews your background against the role.
- 3AssessmentA technical test, assignment, or coding round, depending on the role.
- 4Interview(s)One or more rounds with the hiring team.
- 5OfferOffer letter with compensation and start date.
Before you apply
0/4More at Duolingo
Other jobs at Duolingo
- Proctor Quality And Learning Manager, Scaling Operations · London, England
- Senior Creative, Social · New York, NY
- Ad Sales Lead, Programmatic · New York, NY
- Senior Creative, Social · Pittsburgh, PA
Internships at Duolingo
- Illustrator, Intern · Detroit, MI
Explore Related Placements
// similar opportunities
You might also like
Proctor Quality And Learning Manager, Scaling Operations
Duolingo
Our mission at Duolingo is to develop the best education in the world and make it universally available. It’s a big mission, and that’s where you come in! At Du...
Operations Manager
Flexport
About Flexport: At Flexport, we believe global trade can move the human race forward. That’s why it’s our mission to make global commerce so easy there will be ...
Sr. Product & Design Operations Manager
Pagerduty
PagerDuty, Inc. (NYSE: PD) is the global leader in AI-first digital operations. By automatically detecting, diagnosing, and remediating issues, the PagerDuty Pl...
Commercial Operations Program Manager
Anthropic
About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for s...
Sr. Analyst, Product And Pricing Operations
Salesforce
To get the best candidate experience, please consider applying for a maximum of 3 roles within 12 months to ensure you are not duplicating efforts. Job Category...
Revenue Operations Manager
Scaleai
About the role As Revenue Operations Manager for Enterprise GTM, you will own the measurement layer of our go-to-market engine. You will be the key to what is i...