Greenhouse ·
full-timeStaff Applied Scientist - Agentic Interfaces
Datadog · New York, New York, USA
Team description At Datadog, AI agents are becoming first-class consumers of observability, security, and software delivery data — from third-party coding agents like Claude Code, Cursor, and Copilot, to our own Bits SRE, Bits Assistant, and Bits Dev Agent. The Agentic Interfaces team owns the platform that connects these agents to Datadog: the MCP Server, the tools and retrieval surfaces agents call into, and — critically — the evaluation systems that tell us whether an agent's experience on Datadog data is actually getting better over time. This role is about that last piece. We're hiring a Staff Applied Scientist to define what "good" means for an Agentic interface at Datadog and to build the measurement systems that make it true. "Good" isn't one number — it spans answer quality, tool-selection accuracy, retrieval relevance, latency, token cost, and end-to-end agent success on real customer workflows. You'll design the evals, build the datasets, define the metrics, and partner with the AI engineers on the team to land the platform that lets every product group at Datadog ship integrations that are demonstrably better release over release. The space is full of open research questions. How do you evaluate an agent end-to-end when the trajectory is non-deterministic? How do you score tool selection when the tool catalog has hundreds of entries and grows weekly? How do you build a measurement system that catches regressions across first-party and third-party agents at once, without each team writing their own harness? If those are the problems you want to spend your time on, come build this with us. Datadog values people from all walks of life. We understand not everyone will meet all the above qualifications on day one. That's okay. If you’re passionate about technology and want to grow your skills, we encourage you to apply. What You’ll Do: Own the evaluation strategy for Datadog's AI agent integrations. Define the metrics — offline and online, quality and cost, single-turn and trajectory-level — that the team and the broader organization optimize against. Build the eval datasets, golden traces, and regression harnesses that catch quality changes before they hit customers, and make those assets reusable by every team contributing tools to the platform. Drive measurable improvements to retrieval relevance, tool-selection accuracy, and context efficiency, partnering closely with the AI engineers on the team who build the underlying platform. Run applied research on the open problems in agent–data interaction: tool selection under large catalogs, multi-turn agent evaluation, grounding and hallucination control on live telemetry, cost/quality tradeoffs at scale. Partner with the Bits SRE, Bits Assistant, and Bits Dev Agent teams so first-party agents benefit from the same measurement substrate as third-party integrations, and so learnings move freely in both directions. Provide technical leadership across the Agentic Interfaces team and the broader organization through design reviews, working groups, and mentorship, and represent the team externally through talks, blog posts, and contributions to the open agent ecosystem. Who You Are: You have a BS/MS/PhD in a scientific field, or equivalent experience. 10+ years of relevant engineering or applied science experience, including time as a technical lead. Proven track record of leading ML or GenAI initiatives in a product-driven environment, from research through production. Significant experience with evaluation, experimentation, or measurement of ML systems at scale. You bring a strong product mindset and are comfortable driving initiatives across cross-functional teams. You thrive in ambiguity and can make sound technical calls when the path isn’t yet defined. Benefits and Growth: New hire stock equity (RSUs) and employee stock purchase plan (ESPP) Continuous professional development, product training, and career pathing An inclusive company culture, giving programs, and the ability to join our Community Guilds (Datadog employee resource groups) Competitive global benefits and global Spring Health benefits for employees and dependents age 6+ #LI-Onsite Datadog offers a competitive salary and equity package, and may include variable compensation. Actual compensation is based on factors such as the candidate's skills, qualifications, and experience. In addition, Datadog offers a wide range of best in class, comprehensive and inclusive employee benefits for this role including healthcare, dental, parental planning, and mental health benefits, a 401(k) plan and match, paid time off, fitness reimbursements, and a discounted employee stock purchase plan. The reasonably estimated yearly salary for this role at Datadog is: $276,000 — $345,000 USD About Datadog: Datadog is the leading observability and security platform for the AI era, providing businesses with unified visibility across the technology stack to manage complexity at scale. It brings applications, infras
| Role | Staff Applied Scientist - Agentic Interfaces |
|---|---|
| Company | Datadog |
| Location | New York, New York, USA |
| Type | full-time |
| Compensation | Not disclosed |
| Posted | 2026-09-29 |
| Deadline | Rolling |
Typical process for this type of role
A general guide — the exact steps for this specific listing may vary; check the original posting for details.
- 1ApplicationSubmit your resume through the apply link.
- 2ScreeningRecruiter reviews your background against the role.
- 3AssessmentA technical test, assignment, or coding round, depending on the role.
- 4Interview(s)One or more rounds with the hiring team.
- 5OfferOffer letter with compensation and start date.
Before you apply
0/4About Datadog
Datadog provides a modern monitoring and security platform designed for developers, IT operations teams, and business users operating in the cloud. The company's platform offers a wide range of capabilities, including infrastructure, network, container, and serverless monitoring, as well as application performance monitoring, log management, and cloud security. These tools are utilized by organizations across various industries, such as financial services, healthcare, retail, and technology, to enable digital transformation and drive collaboration across teams. Datadog employs over 8,100 people globally and continues to expand its presence to support customers across diverse markets and regions.
More at Datadog
Other jobs at Datadog
- Senior Group Manager, Experience Design · New York, New York, USA
- Event Marketing Manager - Global Sponsorships (namer/latam) · New York, New York, USA
- Senior Software Engineer, Chaos Engineering · Paris
- Manager, Revenue Accounting · New York, New York, USA
- Field Enablement Manager (emea) · Amsterdam, The Netherlands; Munich, Germany
Internships at Datadog
- Applied Science Intern · Paris, France
- Software Engineering Intern · Paris, France
- IT Support Technician Intern · Paris, France
- Product Management Intern · New York, New York, USA
- Research Science Intern (phd) · New York, New York, USA; Pittsburgh, Pennsylvania, USA
Explore Related Placements
// similar opportunities
You might also like
Staff Applied Scientist - Dashboards
Datadog
The Dashboards product is Datadog's unified single-pane-of-glass for metrics, logs, and traces—a comprehensive treasure trove of observability data. We are tran...
Senior Applied Scientist
Datadog
The Applied AI team designs and builds algorithmically driven features in the Datadog app. We work across a range of applications, primarily focusing on analysi...
Manager I, Applied Science - Frontier Agentic Models
Datadog
Applied AI is where Datadog's ambitious AI bets get built and shipped ( Bits Chat , updog ). We sit at the intersection of research and product: turning promisi...
Staff Applied Scientist
Lyft
At Lyft, our purpose is to serve and connect. We aim to achieve this by cultivating a work environment where all team members belong and have the opportunity to...
Data Scientist II, Applied ML
Brex
Why join us Brex is the intelligent finance platform that enables companies to spend smarter and move faster in more than 200 markets. By combining global corpo...
Senior Data Scientist - Applied ML
Justworks
Who We Are At Justworks, you’ll enjoy a welcoming and casual environment, great benefits, wellness program offerings, company retreats, and the ability to inter...