Talent.com
Mindrift
Freelance Agent Evaluation EngineerMindrift • London, England, GB
Freelance Agent Evaluation Engineer

Freelance Agent Evaluation Engineer

Mindrift • London, England, GB
30+ days ago
Salary
£50.00 hourly
Job type
  • Part-time
  • Permanent
  • Remote
  • Quick Apply
Job description

Please submit your CV in English and indicate your level of English proficiency.

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.

What this opportunity involves

We're building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks.

You'll create challenging tasks and evaluation criteria within realistic simulated environments:

  • Build realistic developer environments - a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history
  • Design tasks from intermediate states of these environments - craft the prompt, define what "solved" means, and ensure the task is solvable by an AI agent
  • Write tests that verify agent solutions - accept all valid approaches and reject incorrect ones, neither too strict nor too lenient
  • Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust

What this is NOT

  • Not data labeling
  • Not prompt engineering
  • Not writing code from scratch - the agent writes most of the code; you guide and evaluate

What we look for

  • 5+ years in software development
  • Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis
  • Experience writing tests (functional, integration)
  • English proficiency - B2+

Why this is hard

Frontier models are already good at coding. Creating a task that genuinely challenges the best models is non-trivial. You need to deeply understand where models fail and what scenarios reveal the difference between a good and a bad solution. Tasks have many valid solutions - writing tests that accept all correct solutions and reject incorrect ones is harder than it sounds.

How it works

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid

Effort estimate

Tasks for this project are estimated to take 20 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.

Compensation

Up to $50/hr equivalent, depending on level and pace. Tasks are estimated at ~20 hours each; you set your own schedule.

Create a job alert for this search

Freelance Agent Evaluation Engineer • London, England, GB

Similar jobs

Senior Forward Deployed Engineer

Just EatLondon, England, GB
Full-time

Senior Forward Deployed Engineer.London or Amsterdam, 3 days a week in office.We're looking for a technically sharp, product-minded engineer to help push the frontier of AI across Just Eat Takeaway... Show more

 • Promoted

AI Evaluation Engineer

Gazelle GlobalGreater London, England, GB
Full-time

Location: London (Hybrid – 1 day onsite).We're looking for an AI Evaluation Engineer to help deliver and optimise cutting‑edge Generative AI solutions within a large enterprise environment.Define a... Show more

 • Promoted

Frontend Engineer — AI Code Agent Evaluator

ObsidianGreater London, England, GB
Full-time

Obsidian is seeking contributors to evaluate and improve frontier AI coding models through structured technical assessments.The role focuses on reviewing AI-generated frontend code and using AI cod... Show more

 • Promoted

Forward Deployed Eng Director

Accenture UKGreater London, England, United Kingdom
Full-time

It is not a project delivery role.A Forward Deployed AI Engineer is a production engineer who works embedded inside a client's enterprise, shoulder to shoulder with their teams, to make complex AI ... Show more

 • Promoted

Senior Forward Deployed Engineer

Just Eat Takeaway.comCity Of London, England, GB
Full-time

Senior Forward Deployed Engineer.London or Amsterdam, 3 days a week in office.We're looking for a technically sharp, product-minded engineer to help push the frontier of AI across Just Eat Takeaway... Show more

 • Promoted

MLOps Engineer

UltralyticsLondon, England, GB
Full-time

At Ultralytics, we relentlessly drive innovation in AI, building the world's leading YOLO models.We're looking for passionate individuals obsessed with AI, eager to make a global impact, and ready ... Show more

 • Promoted

Agentic Customer Engineer

LightGreater London, England, GB
Full-time

Light exists to replace factory-era ERPs with software that feels alive.Our Smart Financial Platform gives modern, global companies superpowers - automated accounting, real-time reporting, and fina... Show more

 • Promoted

Agentic Forward Deployed Engineer

HCLTechGreater London, England, GB
Full-time

HCLTech is a global technology company, home to more than 220,000 people across 60 countries, delivering industry-leading capabilities centered around digital, engineering, cloud and AI, powered by... Show more

 • Promoted

Freelance Product Engineer — AI-Driven Growth for Books

WonderblyGreater London, England, GB
Full-time

Wonderbly is seeking a freelance product engineer to join their team, working 3-4 days per week in a hybrid model.This hands-on role involves collaborating closely with the VP product to address cu... Show more

 • Promoted

Applied AI Engineer - Agentic Systems - Senior Associate

FairygodbossGreater London, England, GB
Full-time

Join us to shape the future of AI-driven business operations.You will have the opportunity to create scalable, safe, and reliable agentic solutions that transform how we work.We value your expertis... Show more

 • Promoted

Implementation Engineer

AdyenGreater London, England, GB
Full-time

Adyen provides payments, data, and financial products in a single solution for customers like Meta, Uber, H&M, and Microsoft - making us the financial technology platform of choice.At Adyen, everyt... Show more

 • Promoted

Backend Engineer for AGI Safety Tools & Evaluations

COL LimitedGreater London, England, GB
Full-time

A pioneering AI research organization is seeking a Backend Software Engineer to develop innovative tools for frontier AGI safety research, focusing on internal tooling and automated evaluation pipe... Show more

 • Promoted

Remote Forward Deployment Solutions Engineer

Unit21Greater London, England, GB
Remote
Full-time

A financial crime prevention firm in Greater London seeks a Senior Forward Deployment Engineer to lead technical solutions and implementations.This hybrid role requires collaboration with Sales and... Show more

 • Promoted

Agent Engineer, TLM

SierraLondon, England, GB
Full-time

At Sierra, we’re creating a platform to help businesses build better, more human customer experiences with AI.We are primarily an in-person company based in San Francisco, with growing offices in A... Show more

 • Promoted

Enterprise AI Deployment Engineer — UK (Hybrid/Remote)

CapitolGreater London, England, GB
Remote
Full-time

A leading AI solutions provider is seeking a Forward Deployed Engineer to join their team in the UK.This hybrid role involves working closely with enterprise clients to design and deploy AI applica... Show more

 • Promoted

Founding Forward Deployed Engineer

Oho GroupGreater London, England, GB
Full-time

Founding Forward Deployed Engineer | AI Start-Up | London.Founding Forward Deployed Engineer required to join a well-backed, early-stage AI start-up in London as Engineer #1.Founded by experienced ... Show more

 • Promoted

Applied AI Engineer - Agentic Systems - Senior Associate

J.P. MorganLondon, England, GB
Full-time

Join us to shape the future of AI-driven business operations.You will have the opportunity to create scalable, safe, and reliable agentic solutions that transform how we work.We value your expertis... Show more

 • Promoted

Forward Deployed Engineer

Arva AIGreater London, England, GB
Full-time

In person, Central London, 4–5 days in office.For exceptional candidates with strong product impact and relevant experience, we are open to offering compensation above the specified range.Arva AI i... Show more

 • Promoted

AI/ML Engineer — RLHF & Model Evaluation (Remote)

ProlificGreater London, England, GB
Remote
Full-time

Prolific is looking for an AI & Machine Learning Engineer to join their team in Greater London.This role involves evaluating LLM architectures, auditing ML code, and providing feedback to enhance m... Show more

 • Promoted

Forward Deployed Engineer

AI Chopping BlockGreater London, England, GB
Full-time

We’re looking for an exceptional Forward Deployed Engineer to help build the future of AI-powered software creation.You will be a founding member of our FDE function, working directly with our most... Show more