Talent.com
Mindrift
Freelance Agent Evaluation EngineerMindrift • Glasgow, Scotland, GB
No longer accepting applications
Freelance Agent Evaluation Engineer

Freelance Agent Evaluation Engineer

Mindrift • Glasgow, Scotland, GB
8 days ago
Salary
£50.00 hourly
Job type
  • Part-time
  • Permanent
  • Remote
  • Quick Apply
Job description

Please submit your CV in English and indicate your level of English proficiency.


Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.


We're building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks.

You'll create challenging tasks and evaluation criteria within realistic simulated environments:

  • Build realistic developer environments - a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history
  • Design tasks from intermediate states of these environments - craft the prompt, define what "solved" means, and ensure the task is solvable by an AI agent
  • Write tests that verify agent solutions - accept all valid approaches and reject incorrect ones, neither too strict nor too lenient
  • Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust

What this is NOT:

  • Not data labeling
  • Not prompt engineering
  • Not writing code from scratch - the agent writes most of the code; you guide and evaluate

What we look for:

  • 5+ years in software development
  • Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis
  • Experience writing tests (functional, integration)
  • English proficiency - B2+

Why this is hard:
Frontier models are already good at coding. Creating a task that genuinely challenges the best models is non-trivial. You need to deeply understand where models fail and what scenarios reveal the difference between a good and a bad solution. Tasks have many valid solutions - writing tests that accept all correct solutions and reject incorrect ones is harder than it sounds.

Requirements and benefits

Educational qualifications

  • A Master’s Degree in Computer Science, Software Engineering, Data Science / Data Analytics, Artificial Intelligence / Machine Learning, Computational Linguistics / Natural Language Processing (NLP), Information Systems or other related fields.
  • Bachelor’s degree is accepted if only candidate has 5 years of experience in the field.

Academic and/or Professional Experience

Candidates should have a minimum of 3 years of professional experience in related roles or domain - specifically for QA-automation/testing or cybersecurity roles

How it works
Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid

Compensation:
Paid per accepted task. Your rate depends on the qualification tier you reach and how efficiently you complete tasks — up to the equivalent of $50/hr. Because payment is per task, a faster pace raises your effective hourly rate.

Why this freelance opportunity might be a great fit for you?

  • Take part in a part-time, remote, freelance project that fits around your primary professional or academic commitments.
  • Work on advanced AI projects and gain valuable experience that enhances your portfolio. - Influence how future AI models understand and communicate in your field of expertise.
Create a job alert for this search

Freelance Agent Evaluation Engineer • Glasgow, Scotland, GB

Similar jobs

Release Train Engineer

N Consulting LimitedGlasgow, Scotland, GB
Full-time

LocationGlasgow, United Kingdom# Release Train Engineer at N Consulting LtdLocationGlasgow, United KingdomSalary£300 - £350 /dayJob TypeContractDate PostedMarch 3rd, 2026Apply NowRelease Train Engi... Show more

 • Promoted

Medical Education SME: AI Content Evaluator

Cactus CommunicationsGlasgow, Scotland, GB
Full-time

Cactus Communications is looking for qualified medical educators in Glasgow, UK, to join their SME team.The role involves evaluating AI-generated medical content to ensure it meets the educational ... Show more

 • Promoted

EY.ai Data Agent Product Owner

EYGlasgow, Scotland, GB
Full-time

At EY, we’re all in to shape your future with confidence.We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go.Join EY and help ... Show more

 • Promoted

Trainee Lifting Engineer

GAP Group LimitedGlasgow, Scotland, GB
Full-time

Our team is the best in the industry – is it time for you to join us? GAP Lifting Hire & Sales offers one of the largest ranges of lifting equipment and accessories in the industry.From small shack... Show more

 • Promoted

Platform Engineer – AWS, Kubernetes & SRE, UK Travel

慨正橡扯Glasgow, Scotland, GB
Full-time

Glasgow or Edinburgh to join our team in delivering excellent digital products.Responsibilities include working directly with clients, applying AI-assisted practices, and ensuring reliable service ... Show more

 • Promoted

Supplier Quality Engineer

Celeros Flow TechnologyGlasgow, Scotland, GB
Full-time

The Supplier Quality Engineer will be responsible for maximizing overall quality and performance of key suppliers, using continuous improvement techniques, while driving out risks.Working in accord... Show more

 • Promoted

Wellsite Performance Engineer

JAB RecruitmentScotland, GB
Full-time

Senior Recruitment Consultant at JAB Recruitment | Engineering and Commercial Recruitment Solutions within the Oil & Gas and Renewables Industries.Location: Offshore (3/3 rotation).An exciting oppo... Show more

 • Promoted

Trainee Site Engineer - England & Wales

United LivingScotland, GB
Full-time

UK-based infrastructure services company delivering critical energy, power, water, telecoms and social infrastructure projects.Following the acquisition of Wood’s UK Transmission & Distribution bus... Show more

 • Promoted

Travel Agent

The Travel ExpertsGlasgow, Scotland, United Kingdom
Full-time +1

Are you passionate about travel and looking for a flexible way to earn income on your own terms? We’re expanding and looking for motivated individuals who want to build their own independent travel... Show more

UK Applied AI Solution Engineer

TomoroGlasgow, Scotland, GB
Full-time

Tomoro enables organisations to realise competitive advantage with the power of Generative AI.We work with large corporate clients to create meaningful AI strategies, build production‑ready AI solu... Show more

 • Promoted

Site Reliability Engineer

Paritas RecruitmentGlasgow, Scotland, GB
Full-time

AWS Site Reliability Engineer (Data Platform) – Contract.Contract Length: February 2026 – January 2027.We are recruiting an AWS Site Reliability Engineer (SRE) to support a cloud-native data platfo... Show more

 • Promoted

Digital Project Controls Engineer: EVM, FAC & Analytics

MatchtechScotland, GB
Full-time

Matchtech is seeking a digitally-enabled Cost / Project Controls Engineer in the United Kingdom, Scotland, to support complex programmes.This role involves utilizing advanced analytics and project ... Show more

 • Promoted

Power Platform Engineer (Hybrid) - Glasgow

Ashurst Perkins CoieGlasgow, Scotland, GB
Full-time

Ashurst Perkins Coie is recruiting a Developer within Labs in Glasgow.The role covers designing, developing and managing digital platforms and client/internal solutions with a mix of support and de... Show more

 • Promoted

Trainee Lifting Engineer

GAP Group Ltd.Glasgow, Scotland, GB
Full-time

United Kingdom, Scotland, Kinning Park Lifting.Number of positions to be provided.Our team is the best in the industry – is it time for you to join us? GAP Lifting Hire & Sales offers one of the la... Show more

 • Promoted

Measurement and Technical Compliance Engineer

Athenia RecruitmentGlasgow, Scotland, GB
Full-time

Measurement and Technical Compliance Engineer Leading Consultancy - Glasgow Competitive salary and package available Salary from £30,000 - £45,000 per annum depending on qualifications and experien... Show more

 • Promoted

OT Engineer

Cyber Fraud CentreGlasgow, Scotland, GB
Full-time

Sword is a leading provider of business technology solutions within the Energy, Public and Finance Sectors, driving transformational change within our clients.We use proven technology, specialist t... Show more

 • Promoted

Lead Site Reliability Engineer - Observability & Reliability

JPMorganChaseGlasgow, Scotland, GB
Full-time

JPMorgan Chase in Glasgow is seeking a Senior Lead Site Reliability Engineer to enhance, build, and deliver critical platforms with top-notch reliability.You will collaborate with an agile team to ... Show more

 • Promoted

Lead Site Reliability Engineer - Observability & Resilience

JPMorgan Chase & Co.Glasgow, Scotland, GB
Full-time

Lead Site Reliability Engineer to define the future of reliability for a global firm.You will lead critical resiliency design reviews, break complex problems into actionable work, and mentor engine... Show more

 • Promoted

Software Engineer (Applied Ai) - £90,000 - £110,000 A Year - Remote

EuphoricGiffnock, United Kingdom Of Great Britain And Northern Ireland, UK
Remote
Full-time

Software Engineer to build AI-powered features for a benefits platform, using React, Python, and cloud platforms. Show more

 • Promoted

Site Reliability Engineer

LorienGlasgow, Scotland, GB
Temporary

Resource Consultant - Public Sector IT & Professional Services.Site Reliability Engineer / Live Operations Engineer - 6 month contract £570 per day OutsideIR35 - Glasgow Hybrid.One of Loriens leadi... Show more