AI Quality Analyst

About Jobsity

Jobsity is a New York–based staffing company with 10+ years of experience. We connect top-tier LATAM talent with long-term international opportunities.

Requirements

• Conversational English level (+B2 level).

• Must reside in LATAM.

• Proven experience in a quality assurance, testing, or data analysis role within the AI/ML domain.

• Deep understanding of the machine learning lifecycle and common AI model failure modes.

• Hands-on experience with data annotation, data validation, and managing large datasets.

• Experience with bug tracking systems (e.g., Jira) and test case management tools.

• Basic scripting skills (e.g., Python) for test automation and data manipulation (Preferred).

Tasks

• Evaluation & Benchmarking: Develop test cases and benchmarks to assess model performance and identify failures.

• Error Analysis & Defect Triage: Investigate model issues, identify root causes, and prioritize fixes.

• Adversarial Testing (Red Teaming): Test models with challenging scenarios to uncover unexpected behaviors and vulnerabilities.

Perks

• Yearly paid vacation.

• Paid local holidays.

• Learning budget.

• Financing program for equipment and home office supplies.

• Bonuses for referrals and more.