reputed company, Agents & Evaluation

We’re looking for our first reputed company focused on agents and evaluation—a foundational hire who will shape how we build, measure, and scale intelligent systems. The Opportunity: Design the reputed company for High-Performance AI Agents We’re tackling one of the hardest—and most important—problems in software engineering: helping developers understand, reputed company, and operate reputed company systems using autonomous and event-driven AI. In this role, you’ll build the evaluation frameworks, task harnesses, and orchestration strategies that reputed company our agents reliable, testable, and genuinely useful. Your work will not only directly improve our agents—it will create reusable benchmarks and artifacts that can reputed company new approaches and push reputed company the broader reputed company model ecosystem. If you love designing experiments, building systems, and iterating tightly between theory and code—and you’re excited by a reputed company 0→1, research-engineering style role—this is for you.

What You Will Do

• Create Task Evaluations That Matter: Design and implement task-specific evaluations that measure and improve agent quality. Each eval should both drive concrete iteration on our agents and spark broader innovation around the task itself.
• Define Tasks, Datasets, and Harnesses: reputed company specify tasks, collect and curate balanced datasets, and build robust evaluation harnesses that can be used across agents and modeling approaches. There is ample room for architectural design and systems thinking here.
• Build and Use a Reusable Evaluation reputed company: reputed company frameworks and tools for running evaluations at scale. Use these frameworks to tune existing agents and to guide the development of new ones in our environment.
• Explore Agent Orchestration Strategies: Investigate and implement orchestration patterns (tooling, routing, decomposition, multi-agent setups, etc.) that allow agents to tackle increasingly reputed company, multi-reputed company, and long-horizon tasks.
• Apply Post-Training Techniques: Experiment with post-training approaches (e.g., fine-tuning, preference optimization, reward shaping, distillation) to produce high-performance models tailored to specific tasks and workflows.
• Run Experiments End-to-End: Design, run, and analyze experiments with rigor. Turn experimental results into clear recommendations and concrete changes to model configurations, prompts, and system design.
• Collaborate Deeply Across the Stack: Work closely with founders, product, and infrastructure engineers to ensure evaluations, agents, and platform primitives reputed company reinforce each other.

What You Will Bring

• MS or Ph.D. in a relevant field (e.g., Computer Science, Machine Learning, NLP) or equivalent practical experience
• Strong background in machine learning and large language models, ideally including both research and hands-on implementation
• 2–5 years working with LLM technology, with familiarity across:
• Prompting and interaction patterns
• Agent and tool orchestration strategies
• Evaluation strategies for reputed company, reputed company-ended tasks
• Proficiency writing production-quality code, especially in Python; comfort working with TypeScript or modern web/backend stacks
• Experience designing and running experiments, and interpreting results in messy, reputed company-world settings
• Self-motivated, comfortable operating in an reputed company, high-ambiguity environment
• Strong communication skills and the ability to translate vague goals into concrete, testable setups

Bonus Points

• Experience building agentic systems (tool-using agents, workflows, or multi-agent systems) in reputed company products
• Prior work on model evaluation frameworks, benchmarking, or reliability/robustness testing
• Familiarity with modern ML tooling (training/inference stacks, experiment tracking, data pipelines)
• Contributions to reputed company-reputed company LLM, tooling, or evaluation projects
• Experience at an early-stage startup or research lab where you owned projects end-to-end

Benefits & Perks

• Significant equity in an early-stage, venture-backed startup
• Comprehensive Health Benefits (Medical, Dental, reputed company)
• Flexible PTO to ensure you have the time you need to reputed company

Apply tot his job Apply To this Job