AI Quality Analyst

About Jobsity

Jobsity is a New York–based staffing company with 10+ years of experience. We connect top-tier LATAM talent with long-term international opportunities.

Requirements

• Conversational English level (+B2 level).
• Must reside in LATAM.
• Proven experience in a quality assurance, testing, or data analysis role within the AI/ML domain.
• Deep understanding of the machine learning lifecycle and common AI model failure modes.
• Hands-on experience with data annotation, data validation, and managing large datasets.
• Experience with bug tracking systems (e.g., Jira) and test case management tools.
• Basic scripting skills (e.g., Python) for test automation and data manipulation (Preferred).

Tasks

• Evaluation & Benchmarking: Develop test cases and benchmarks to assess model performance and identify failures.
• Error Analysis & Defect Triage: Investigate model issues, identify root causes, and prioritize fixes.
• Adversarial Testing (Red Teaming): Test models with challenging scenarios to uncover unexpected behaviors and vulnerabilities.

Perks

• Yearly paid vacation.
• Paid local holidays.
• Learning budget.
• Financing program for equipment and home office supplies.
• Bonuses for referrals and more.