Europe Arbeitnow ·
full-timePrincipal AI Evaluations Platform Engineer
Microsoftcorporation · London
As Microsoft continues to push the boundaries of AI, we are looking for passionate individuals to work with us on some of the most interesting and challenging AI questions of our time. Our vision is bold and broad: to build systems with true artificial intelligence across agents, applications, services, and infrastructure. It is also inclusive: we aim to make AI accessible to consumers, businesses, and developers so that everyone can realize its benefits. Microsoft AI (MS AI) is seeking an experienced engineer to help build and operate the evaluation platform that supports large-scale model training and development. We’re looking for someone who combines strong systems engineering fundamentals with operational excellence and who can build reliable evaluation infrastructure at scale. This role will work closely with researchers and training teams across our European offices to ensure evaluations are reliable, efficient, and available when teams need them. We seek a versatile engineer who can build solutions that stand the test of time and who brings positive energy, empathy, and kindness to the team while remaining highly effective in a fast-paced environment. Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees, we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day, we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. This role is based in London, U.K., or Zurich, Switzerland. Candidates are expected to be local to the applicable office and work in the office four days per week. Responsibilities Serve as a primary engineer for the evaluation platform during European hours, providing dedicated coverage for the evaluation stack supporting large-scale model training. Develop and extend core evaluation platform capabilities, including benchmark configurations, problem sets, graders, and evaluation runners. Improve evaluation throughput and scheduling efficiency. Build and maintain monitoring, alerting, and dashboards to proactively detect evaluation degradation. Partner closely with researchers and training teams across European offices to define evaluation requirements, coordinate launches, and unblock experiments. Contribute to technical design and code reviews across the evaluation platform stack, and maintain high-quality operational documentation and on-call runbooks. Embody our culture and values. Required Qualifications Bachelor's Degree in Computer Science, Engineering, Mathematics, or a related field AND 4+ years of technical engineering experience with coding in languages including, but not limited to, Python, C++, Go, or Rust; OR Master's Degree in Computer Science, Engineering, Mathematics, or a related field AND 2+ years of technical engineering experience with coding in languages including, but not limited to, Python, C++, Go, or Rust; OR equivalent experience. Demonstrated experience operating production distributed systems, including participation in a formal on-call rotation. Strong proficiency in Python, including experience working in a large, shared, typed codebase. Experience debugging distributed system failures across the stack, including scheduling, resource exhaustion, networking, and process-level faults. Willingness and availability to participate in a scheduled on-call rotation covering European business hours. Preferred Qualifications Experience with GPU-based workloads, distributed training, or large-scale inference infrastructure. Experience with distributed compute frameworks and schedulers, such as Ray, Kubernetes, Slurm, or comparable systems. Experience with observability and alerting tooling, such as Datadog, Prometheus, or Grafana. Familiarity with LLM evaluation, benchmarking, or reinforcement learning training infrastructure. Track record of reducing operational burden through systematic reliability improvements, automation, or tooling. Ability to work effectively in a fast-paced environment, manage multiple priorities, and adapt to changing requirements and deadlines. The typical base pay range for this role across Switzerland is CHF 183,800.00 - CHF 309,700.00 per
| Role | Principal AI Evaluations Platform Engineer |
|---|---|
| Company | Microsoftcorporation |
| Location | London |
| Type | full-time |
| Compensation | Not disclosed |
| Deadline | Rolling |
Typical process for this type of role
A general guide — the exact steps for this specific listing may vary; check the original posting for details.
- 1ApplicationSubmit your resume through the apply link.
- 2ScreeningRecruiter reviews your background against the role.
- 3AssessmentA technical test, assignment, or coding round, depending on the role.
- 4Interview(s)One or more rounds with the hiring team.
- 5OfferOffer letter with compensation and start date.
Before you apply
0/4Explore Related Placements
// similar opportunities
You might also like
Lead, Platform Engineer
Quantum
Our Company At Quantum, we connect global brands with high-intent consumers. We turn intent into action and complexity into clarity. Our Mission is clear, to he...
Principal Engineer - Backend
Encord
About us Encord is the universal data layer for AI that helps 300+ AI teams train and run models on the right data. Our platform indexes, curates, annotates, an...
Database Platform Engineer
Zopa
Our Story Hello there. We’re Zopa. We started our journey back in 2005, building the first ever peer-to-peer lending company. Fast forward to 2020 and we launch...
Principal AI Engineer
Mitratech
Principal AI Engineer (AI / .NET Core / React) At Mitratech, we’re a team of engineers passionate about building world-class products that empower Legal, Risk, ...
Backend / Data Platform Engineer
Foundational
Backend / Data Platform Engineer This is a future role. We're building a shortlist now and will contact you when it opens. About us Foundational is building the...
Principal Software Engineer (hybrid)
8thlightrebuild
Principal Engineer Location : Waterloo, London, England, UK | Hybrid (1 day per week in office) Who We Are At 8th Light, we bring life to the best ideas. Founde...