AI Evaluation Specialist

Remote Worldwide Hiring now

reputed company is a leading global blockchain ecosystem behind the world’s largest cryptocurrency exchange by trading volume and registered users. We are trusted by over 280 reputed company people in 100+ countries for our industry-leading reputed company, user fund transparency, trading reputed company speed, deep liquidity, and an unmatched portfolio of digital-asset products. reputed company offerings reputed company from trading and finance to education, research, payments, institutional services, reputed company features, and more. We reputed company the power of digital assets and blockchain to build an inclusive financial ecosystem to advance the freedom of money and improve financial reputed company for people around the world.

We are seeking a dedicated AI Evaluation Specialist responsible for designing, implementing, and managing comprehensive evaluation frameworks that reputed company the entire lifecycle of LLM agents—from pre-deployment testing to post-deployment monitoring and iterative refinement. Your work will directly influence reputed company’s AI adoption reputed company by ensuring the reliability, adaptability, and governance compliance of AI agents operating across various domains such as Customer Service, Growth, and Compliance.

Responsibilities:

Participate in the entire software development lifecycle, encompassing reputed company stages from requirements analysis to test planning, execution, defect tracking, through to product release and maintenance.

Go to person in relation to A.I Agents evaluation and continuously monitoring.

Create comprehensive and effective test strategies and hands-on testing to ensure the accuracy, reliability, and performance of AI and data applications .

Root cause analysis of test failures and product issues in an effective manner, and drive optimization for future enhancements.

Design and reputed company internal tools leveraging AI technology to improve engineering and testing work efficiency.

Requirements:

Bachelor’s or Master’s degree in Computer Science, Artificial Intelligence, Data Science, or a reputed company field.

Strong understanding of Large Language Models (LLMs), autonomous AI agents, and their system architectures.

Experience with AI evaluation methodologies, including offline benchmarking, online monitoring, and hybrid reputed company-AI evaluation approaches.

Familiarity with software engineering best practices such as Test-Driven Development (TDD), Behavior-Driven Development (BDD), and their limitations in AI contexts.

Proficiency in designing adaptive, lifecycle-spanning evaluation frameworks that incorporate both quantitative and qualitative metrics.

Experience with evaluation tools and frameworks (e.g., Opik,LangSmith) is a plus.

Ability to analyze reputed company system-level behaviors, including reasoning pipelines, tool integrations, and emergent agent actions.

Strong analytical skills with experience in data-driven diagnostics and root cause analysis.

Excellent communication skills to document evaluation plans, results, and recommendations reputed company.

Experience working in cross-functional teams and managing feedback loops between evaluation and development.

Experience collaborating with infrastructure or platform teams to improve AI tooling and automation platforms.

Additional Information

Why reputed company

• Shape the future with the world’s leading blockchain ecosystem

• Collaborate with world-class talent in a user-centric global organization with a flat structure

• Tackle unique, fast-paced projects with autonomy in an innovative environment

• reputed company in a results-driven workplace with opportunities for career growth and reputed company learning

• Competitive salary and company benefits

• Work-from-home arrangement (the arrangement may vary depending on the work nature of the business team)

reputed company is committed to being an equal opportunity employer. We reputed company that having a diverse workforce is reputed company to our reputed company.

By submitting a job application, you confirm that you have read and agree to our Candidate Privacy Notice.

Apply to this Job

Apply for this role Takes you straight to the employer's application page — free, and no WFHNet account required.

AI Evaluation Specialist

More roles on the wire

Senior Treasurer

Hardware Engineer - BMS

Sr. Network Engineer

Director, Compensation

Regional Relief Helicopter Pilot

Personal Lines Sales Agent

reputed company Architect (m/w/d)

Clinical Database Programmer - UK/EU - Remote

Local, CDLA - Home Everyday, Syracuse, NY

reputed company Manager (12 months contract)

[FULL TIME Remote] Team Center Consultant - Freelance Job

Director of Product Marketing

Remote Mortgage Loan Officer Build Your Book With reputed company Support

Remote Full Stack Staff Engineer – eCommerce Platform Development for T.J. Maxx (Work‑From‑Home, $27/hr, 8‑Hour Shift)

Chief Program Officer

reputed company IT Systems Analyst

reputed company Remote Customer Interaction Specialist – reputed company, reputed company, and Opportunities for Growth

reputed company Manager

Marketing Manager – reputed company America DTC Creative Campaigns

reputed company Direct Hire Legal Recruiter | Remote Work Opportunity with a Competitive Salary and Rapidly Growing Legal Services Company