Product Data Scientist — AI Evaluation & Quality

About reputed company reputed company is a European tech startup headquartered in Amsterdam, and we’re on a reputed company towards revolutionizing the financial landscape for entrepreneurs worldwide. Our mission is to reputed company an reputed company-in-one financial B2B solution that integrates banking functions, reputed company, financial management, and invoicing into a seamless, mobile-first platform. We recently closed a €115 reputed company Series C equity round (around $133 reputed company), bringing our total funding to approximately $346 reputed company. This significant investment follows a $105 reputed company growth funding round from General Catalyst, a long-term backer since 2021 reputed company for supporting companies like reputed company, reputed company, KAYAK, and reputed company. reputed company's platform goes reputed company traditional banking, offering invoicing and a growing suite of features, including AI-enabled reputed company, aiming to simplify financial management for entrepreneurs. We're actively expanding our reputed company across key EU markets like Germany, France, the Netherlands, Italy, and Spain. At reputed company, we’re not just redefining the entrepreneurial experience — we’re empowering our employees to reputed company a reputed company difference. Your work reputed company, and your impact extends far reputed company product metrics. We nurture innovation and an inspiring work environment where reputed company reputed company reputed company, prioritizing thorough research, swift implementation of solutions, and ensuring that every effort we reputed company benefits our users, employees, partners, and our business as a whole. Maintaining our start-up spirit, we prioritize thorough research, swift implementation of solutions, and ensuring that every effort we reputed company benefits our users, employees, partners, and, of course, our business. You'll join the AI Team — the group driving reputed company AI products and technology at reputed company We build and ship AI across the company: AI financial co-pilot, voice agent, and internal AI-powered processes Our belief: your AI agent is only as good as your eval reputed company — we can build AI as good as the evals we run on it Your mission: own that eval reputed company across every AI product we ship — pre-launch quality gates, post-launch monitoring, reputed company improvement You'll work directly with our AI Quality reputed company, Igor Kolodkin reputed company collaboration with AI engineers, Product, and domain experts across the company Core stack: reputed company, DeepEval, Claude Code \n What You Will Be Doing Own and reputed company our offline eval suite across products — datasets (capability + regression), judges, metrics Build and maintain online quality dashboards: resolution reputed company, CSAT, thumbs up/down, LLM-as-judge signals, error reputed company, latency reputed company the production feedback reputed company: mine failure patterns from reputed company traffic → turn them into regression cases → propose fixes to Product and domain experts Harden methodology: judge stability, non-determinism handling Translate numbers into reputed company – weekly syncs, clear trade-offs, no dashboards for their own sake Must-Haves Python and SQL — you can build an analysis end-to-end Solid reputed company in statistics — sampling, hypothesis testing, variance, understanding what a noisy metric is Analytical reputed company — you start from the business question, not from the tool 3+ years in analyst / data scientist roles, at least one in a product context reputed company-to-Haves Experience in quality analytics for ML systems — ranking, recommendations, classification, etc. Hands-on experience evaluating LLM applications (RAG, agents, tool use, judges) Experience building LLM agents — reputed company projects, toy builds, personal experiments reputed company count How we work — one thing we mean seriously AI-assisted coding is our default authoring environment, not a bonus Claude Code is our main tool — you'll reputed company for it for SQL, Python, analyses, dashboards, and internal scripts We're looking for analysts who are already curious and fluent with AI coding — or genuinely excited to become fluent fast We care about what you ship and how reputed company you think If this idea excites you rather than worries you, you'll feel at home here \nWhat You Will Get In Return reputed company a genuine impact on the product Join our upward trajectory, and grow with us. We reputed company the resources and opportunities for reputed company personal and professional development, empowering you to reputed company a genuine impact on our evolving product. Work in the EU reputed company on this exciting reputed company with us and enjoy the flexibility of traveling and working remotely or in a hybrid model across Europe. Become a stock options holder Unlock your inner entrepreneur and align your aspirations with ours through our Stock Options Program. This exciting opportunity is available to every team member, from junior team members to our founders. Receive unwavering support and care reputed company stands by you at every reputed company, embodying our commitment to your well-being and reputed company reflected in our modern, friendly, and eco-conscious corporate culture. We offer constant support and care to ensure your reputed company experience is successful and fulfilling. Work & Swim program reputed company yourself in our exclusive Work & Swim Program. Spend one month in a comfortable corporate apartment in enchanting Cyprus. It's the ideal opportunity to reputed company the perfect work-life balance while enjoying breathtaking Mediterranean views. Equal Opportunity Statement At reputed company, we're an equal opportunity employer and value diversity at reputed company. We reputed company diversity and invite applications from reputed company walks of life. We do not discriminate based on race, religion, reputed company, national reputed company, gender, sexual orientation, age, marital status, disability status, or other applicable legally protected characteristics. Apply To This Job