Senior Data Labeling Specialist (Vancouver)

Senior Data Labeling Specialist (Vancouver) — Remote (Full-Time)

Senior data labeling roles power AI/ML training workflows by producing high-quality training data for LLMs and computer vision models through data annotation, RLHF preference ranking, QA evaluation, and guideline-driven review across real production pipelines.

About The Role

You will lead and execute senior-level data labeling and review work that improves model performance for large language models and vision systems. You will annotate and audit datasets, run QA evaluation against annotation guidelines, perform RLHF preference ranking for response quality, and deliver structured feedback to improve training data quality.

Work includes prompt evaluation, named entity recognition, content safety labeling, and computer vision annotation (bounding boxes, polygons, segmentation) depending on project needs.

What You Will Do

• Own end-to-end labeling execution, quality auditing, and escalation handling for ambiguous edge cases in NLP and CV tasks
• Apply taxonomy definitions, maintain decision logs, and measure inter-annotator agreement
• Support RLHF and evaluation pipelines by ranking outputs, identifying failure modes, and tagging safety/policy violations
• Help define gold-standard sets, build rubrics for QA evaluation, and improve guideline compliance across distributed teams
• Collaborate with engineers and model stakeholders to resolve ambiguity and calibrate decisions
  
  

Requirements

• Strong experience in data labeling/data annotation with documented QA evaluation responsibility
• Ability to interpret complex annotation guidelines and apply consistent judgment at high throughput
• Familiarity with RLHF, prompt evaluation, named entity recognition, computer vision annotation, and content safety labeling
• Comfort working independently in a remote environment and communicating edge cases clearly
  
  

How To Apply

Apply via Rex.zone and include examples of guideline interpretation, QA outcomes, and any RLHF or prompt evaluation work. Highlight domains (NLP, computer vision, content safety) and any metrics (agreement rates, defect reductions, throughput vs. quality targets).