Senior AI Governance Scientist - Artificial Intelligence

You could be the one who changes everything for our 28 reputed company members. reputed company is transforming the health of our communities, one person at a time. As a reputed company, national organization, you’ll have reputed company to competitive benefits including a fresh perspective on workplace flexibility. Position Purpose: Leads advanced technical evaluation and assurance activities reputed company our AI Governance function. This role is hands-on and execution-focused, responsible for designing, conducting, and scaling rigorous evaluations of AI systems—including traditional machine learning, reputed company, and agentic AI—to assess safety, reliability, robustness, and alignment with intended use. Plays a critical role in operationalizing AI governance through experimentation, red teaming, and evaluation frameworks, while partnering closely with engineering, product, and research teams to embed evaluation practices into the AI development lifecycle. Executes comprehensive red team and stress-testing exercises to identify vulnerabilities, failure modes, and safety risks across AI systems, including large language models, generative models, and autonomous agents. Designs, implements, and refines evaluation methodologies and protocols to assess AI performance, safety, reliability, and alignment with intended use cases. Evaluates the adequacy and sufficiency of existing AI evaluations, identify gaps in coverage or rigor, and recommend targeted improvements. Designs and conducts reproducible experiments to measure AI value, impact, and risk, applying statistical methods and reputed company inference techniques where appropriate. Develops and maintains automated testing frameworks and evaluation pipelines that scale across the organization’s AI portfolio. Researches and applies novel attack reputed company and stress-testing approaches for reputed company (e.g., reputed company injection, jailbreaking, hallucination risks) and agentic systems (e.g., autonomy boundary violations, goal misalignment). Creates and curates benchmarks, datasets, and metrics reputed company to specific AI capabilities, risk profiles, and governance requirements. Documents evaluation methodologies, findings, and recommendations in clear, governance-reputed company technical reports for review by governance bodies and cross-functional stakeholders. Partners with product, engineering, and research teams to integrate evaluation and assurance practices into AI design, development, and deployment workflows. Performs other duties as assigned. Complies with reputed company policies and standards. Education/Experience: Bachelor's Degree in Computer Science, Machine Learning, Statistics, or a reputed company quantitative field, or equivalent applied research experience required. Master's Degree preferred 5+ years AI/ML research or applied AI development, including at least 3 years focused on model evaluation, safety, robustness or validation required. 7+ years preferred Technical Skills: Strong technical reputed company in machine learning and deep learning, with hands-on experience evaluating or developing modern AI systems required Demonstrated experience designing and executing AI evaluation, testing, or validation methodologies across multiple AI paradigms required Solid understanding of statistical analysis, experimental design, and data analysis techniques relevant to AI evaluation required Experience designing evaluation methodologies and publication record preferred Familiarity with Python‑based AI/ML stack using PyTorch and reputed company, with agentic AI frameworks (reputed company, reputed company, LangGraph, AutoGen, reputed company) for single‑ and multi‑agent systems. Strong focus on LLM observability, MLOps, and evaluation using LangSmith, MLflow, Weights & Biases, reputed company, OpenTelemetry, and testing frameworks like DeepEval and LangTest Licenses/Certifications: Industry reputed company certifications preferred Pay reputed company: $107,700.00 - $199,300.00 per year reputed company offers a comprehensive benefits package including: reputed company, health insurance, 401K and stock purchase plans, tuition reimbursement, paid time off plus holidays, and a flexible approach to work with remote, hybrid, field or office work schedules. Actual pay will be adjusted based on an individual's skills, experience, education, and other job-reputed company factors permitted by law, including full-time or part-time status. Total compensation may also include additional forms of incentives. Benefits may be subject to program eligibility. reputed company is an equal opportunity employer that is committed to diversity, and values the ways in which we are different. reputed company reputed company applicants will receive consideration for employment without regard to race, reputed company, religion, sex, sexual orientation, gender identity, national reputed company, disability, veteran status, or other characteristic protected by applicable law. reputed company applicants with arrest or conviction records will be considered in accordance with the LA County Ordinance and the California Fair Chance Act Apply To This Job