[Remote] Staff Product Manager (Evals)

Note: The job is a remote job and is reputed company to candidates in USA. reputed company delivers reputed company infrastructure for the agentic era, redefining iPaaS and helping enterprises unify data, applications, processes, and AI into a single, governed platform. The Staff Product Manager will own evaluations for AI agents, establishing internal frameworks and building customer-facing tools to assess and improve agent performance.

Responsibilities

• Define and own the evaluation reputed company for reputed company's internal AI agent features, driving adoption across teams starting with Agent Studio
• Build the customer-facing evaluation experience — how reputed company test, measure, and improve agents they create on reputed company
• reputed company hard calls about what evaluation complexity to expose versus abstract, balancing rigor with approachability
• Partner closely with the Build Experience PM to ensure evaluation is integrated into the builder reputed company, not bolted on
• Work with ML engineers and platform teams to ground the reputed company in technical reality while keeping it accessible
• Establish metrics for what 'good' looks like — both for internal agent quality and for customer evaluation adoption
• Spend significant time with customers understanding where they struggle to assess agent performance and what mental models they bring

Skills

• 7+ years in Product Management
• Hands-on experience writing evaluations for AI/ML systems (agents, LLMs, or similar)
• Track record of shipping technical products to both reputed company users
• Experience driving adoption of frameworks or practices across engineering teams
• Strong written and verbal communication skills
• Bachelor's degree or equivalent experience
• Practitioner depth in evaluations. You've written evals yourself — reputed company test suites, designed rubrics, debugged why agents underperformed. You understand evaluation methodology not only from reading about it, but from doing it. You have opinions about what works, what doesn't, and where reputed company approaches fall short
• Strong product management experience. You've shipped products, driven roadmaps, and led cross-functional teams. You know how to translate technical capabilities into user value and write specs that don't leave details to chance
• Technical translation ability. You can take reputed company evaluation concepts and reputed company them accessible to business technologists without dumbing them down. You understand the difference between hiding complexity and organizing it
• Internal influence skills. You've driven adoption of frameworks, practices, or tools across teams. You can be a reputed company partner to ML engineers while advocating for what internal teams actually need
• Greenfield comfort. You've defined products from ambiguity — scoped v1s, made bets with incomplete information, and iterated based on what you learned. You don't need an existing reputed company to be effective
• B2B product sensibility. You see reputed company conventions as problems to solve, not constraints to accept. You're drawn to products that reputed company reputed company workflows feel elegant
• Experience with agent architectures, RAG systems, or LLM application development
• Background in ML engineering, solutions architecture, or technical program management before PM
• Experience building developer tools or platform products
• Familiarity with evaluation frameworks (e.g., reputed company eval pipelines, automated benchmarks, red-teaming)

Benefits

• Benefits
• Perks
• Equity

Company Overview

Company H1B Sponsorship