Head of Delivery

Head of Delivery, Human Data & Creative AI Evaluation

Salary: $200k - $250k + bonus

Location: New York or SF - Hybrid

Full-time

Morpheus are partnered with a frontier human data and evaluation lab for creative AI, backed by top-tier VCs. They draw on a network of over a million independent creatives to run expert evaluation, preference data, benchmarking and RL environment work across design and creative domains, where expert judgment is the product. They work with leading AI labs and creative-tool companies.

The role

As Head of Delivery, you will own delivery across every client project and lead a team of Strategic Project Leads. You will set how projects are scoped, staffed, reported and measured, and you will be the person clients rely on when delivery needs attention.

This is a player-coach seat. Early on, you will also run one or two of the most complex accounts yourself, with direct access to the Head of the Lab.

What you’ll do

• Lead, hire and coach the Strategic Project Lead team, and decide how they are staffed across accounts


• Own on-time, high-quality delivery across expert evaluation, annotation and benchmarking projects


• Build and run a consistent operating cadence, including weekly delivery reports and monthly forecasts for every client


• Review handling-time estimates and cost models before they go into proposals


• Write and maintain SOPs for expert onboarding, calibration, quality review and payments


• Track and improve portfolio metrics: quality and rater agreement, on-time delivery, handling time against estimate, and margin


• Act as the escalation point for client delivery issues


• Partner with research on methodology and with engineering on tooling

What you bring

• 5+ years in operationally intensive roles, including time at a human data or AI training data company (think Scale AI, Surge AI, Mercor, Handshake AI, Turing or Snorkel)


• 2+ years managing people


• Experience running multiple client projects at once, not only a single project


• Working knowledge of annotation quality: rubric design, calibration and inter-rater agreement


• Ability to build a cost model from handling times and rates


• Comfort working directly with research teams at AI labs

Nice to have

• Creative or design domain experience, or experience with subjective and taste-based evaluation


• A background in consulting, banking or marketplace operations


• Experience building operating processes from scratch in an early-stage team