Talent.com
Mindrift
Senior Python Engineer - AI Coding Agent Evaluation (Freelance)Mindrift • Glasgow, Scotland, GB
Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

Mindrift • Glasgow, Scotland, GB
24 days ago
Job type
  • Part-time
  • Permanent
  • Remote
  • Quick Apply
Job description

Please submit your CV in English and indicate your level of English proficiency.


Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.What this opportunity involves:
We're building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks.


You'll create challenging tasks and evaluation criteria within realistic simulated environments:

  • Build realistic developer environments - a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history
  • Design tasks from intermediate states of these environments - craft the prompt, define what "solved" means, and ensure the task is solvable by an AI agent
  • Write tests that verify agent solutions - accept all valid approaches and reject incorrect ones, neither too strict nor too lenient
  • Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust

What this is NOT:

  • Not data labeling
  • Not prompt engineering
  • Not writing code from scratch - the agent writes most of the code; you guide and evaluate

What we look for:

  • 8+ years in software development
  • Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis
  • Experience writing tests (functional, integration)
  • English proficiency - B2+

Why this is hard:
Frontier models are already good at coding. Creating a task that genuinely challenges the best models is non-trivial. You need to deeply understand where models fail and what scenarios reveal the difference between a good and a bad solution. Tasks have many valid solutions - writing tests that accept all correct solutions and reject incorrect ones is harder than it sounds.

How it works
Apply → Pass qualification(s) → Join a project → Complete tasks → Get paidEffort estimate


Tasks for this project are estimated to take 30 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.

Compensation:
Up to $200/hr equivalent, depending on level and pace. Tasks are estimated at ~30 hours each; you set your own schedule.

Create a job alert for this search

Senior Python Engineer - AI Coding Agent Evaluation (Freelance) • Glasgow, Scotland, GB

Similar jobs

Senior Python AI Backend Engineer

JPMorgan Chase & Co.Glasgow, Scotland, GB
Full-time

JPMorganChase in Glasgow is seeking a Software Engineer III to build production software with a focus on AI-enabled capabilities.You will develop Python backends, APIs, and microservices, and imple... Show more

 • Promoted

Senior ML Engineer: Banking AI & Cloud Deployments

BarclaysGlasgow, Scotland, GB
Full-time

A leading financial services company in Glasgow is seeking a Senior Machine Learning Engineer to enhance their AI/ML initiatives.You will build and manage data architectures while working with clou... Show more

 • Promoted

Senior Enterprise Account Executive - Advanced Computing (HPC & AI) - UK & Nordics

sghcorp.comEast Kilbride, Scotland, GB
Full-time

Senior Enterprise Account Executive - Advanced Computing (HPC & AI) - UK & Nordics.Location: East Kilbride, GB, G75 0QD.Penguin Solutions is seeking a highly motivated and experienced Sr.Enterprise... Show more

 • Promoted

AI Platform Engineer

hackajobGlasgow, Scotland, GB
Full-time

Join Us as an AI Platform Engineer – Shape the Future of AI at Barclays.We’re excited to launch a groundbreaking initiative at Barclays – building a next‑generation platform that empowers front‑off... Show more

 • Promoted

Senior Full-Stack AI Tooling Engineer

Limelight HealthGlasgow, Scotland, GB
Full-time

Barclays is seeking experienced Software Engineers to enable the adoption of industry-standard Generative AI development tools across the enterprise.You will build infrastructure, supporting applic... Show more

 • Promoted

Senior Python Developer – Finance Risk & CI/CD (Glasgow)

Work2getherGlasgow, Scotland, GB
Permanent

Work2gether is seeking a Python Developer based in Glasgow.This role involves utilizing over 5 years of strong Python experience and skills in Object Oriented Programming (OOP).Candidates should be... Show more

 • Promoted

Senior Lead Software Engineer - Python

JPMorgan ChaseGlasgow, Scotland, GB
Full-time

As a Senior Lead Software Engineer at JPMorgan Chase within the Corporate & Investment Bank Post Trade Technology team, you are an integral part of an agile team that works to enhance, build, and d... Show more

 • Promoted

Lead Forward‑Deployed AI Engineer

EYGlasgow, Scotland, GB
Full-time

EY in Glasgow seeks an Associate Director to lead the design, development, and deployment of AI-driven, data-centric solutions for strategic client environments.You will mentor engineers, collabora... Show more

 • Promoted

Senior Software Engineer - Lead AI Mobile Apps

OVO GroupGlasgow, Scotland, GB
Full-time

OVO Group in the UK is seeking a Senior Software Engineer to own the end-to-end development of complex systems and features that are core to our business.You will lead technical initiatives, mentor... Show more

 • Promoted

Applied AI Solutions Engineer — Hybrid, LLM & Impact

TomoroGlasgow, Scotland, GB
Full-time

A leading AI solutions provider in Glasgow is looking for an Applied AI Solution Engineer to design, build, and deploy AI applications using large language models.The role includes working directly... Show more

 • Promoted

Senior SDET: REST & Python Automation (Hybrid)

SASGlasgow, Scotland, GB
Full-time

Software Development Engineer in Test-.Through our software and services, we inspire customers around the world to transform data into intelligence - and questions into answers.We’re also a debt-fr... Show more

 • Promoted

Remote Backend SaaS Engineer — Python & APIs

CanonicalGlasgow, Scotland, GB
Remote
Full-time

A leading software company is looking for a Software Developer (Backend SaaS) to work remotely in the Americas region.This position involves collaborating with a distributed team to develop and mai... Show more

 • Promoted

Senior Python Backend Engineer — APIs & Architecture

CapgeminiGlasgow, Scotland, GB
Full-time

A leading global technology firm is seeking a skilled Python Developer based in Glasgow.The role involves designing, developing, and maintaining Python interfaces and backend services, ensuring cod... Show more

 • Promoted

Lead Python Engineer - Secure Cloud-Native Finance Tech

J.P. MorganGlasgow, Scotland, GB
Full-time

A leading global financial services firm in Glasgow is seeking a Lead Software Engineer to join their Emerging Market Repo Trade Management Technology team.This role involves enhancing and deliveri... Show more

 • Promoted

Senior Platform Engineer (Data Streams) (Remote - UK)

Yelp Inc.Glasgow, Scotland, GB
Remote
Full-time

We’re looking for experienced engineers to join our team and build elegant, scalable platforms that bring the potential of data streams to Yelp’s features teams.From transporting events reliably be... Show more

 • Promoted

AI/ML Engineer — RLHF & Model Evaluation (Remote)

ProlificGlasgow, Scotland, GB
Remote
Full-time

Prolific is seeking an AI & Machine Learning Engineer to evaluate LLM architectures and refine ML frameworks.The ideal candidate will have a strong educational background in Computer Science or a r... Show more

 • Promoted

Senior Python & Databricks Engineer

Stars ArenaGlasgow, Scotland, GB
Full-time

Stars Arena is seeking a Software Engineer III with advanced proficiency in Python and familiarity with Databricks.The role involves delivering system design, application development, testing, and ... Show more

 • Promoted

Python Developer

Venesky BrownGlasgow, Scotland, GB
Temporary

Venesky-Brown’s client, a public sector organisation in Edinburgh / Glasgow, is currently looking to recruit a Python Engineer for an initial 6 month contract with potential to extend on a rate of ... Show more

 • Promoted

Senior UI Engineering Lead - React, Cloud & AI

Next Frontier CapitalGlasgow, Scotland, GB
Full-time

JPMorganChase, within Markets Technology, is looking for a Lead Software Engineer who will be a core technical contributor and integral to an agile team delivering trusted market-leading technology... Show more

 • Promoted

Python Developer for Fixed Income & Core Architecture

PurviewGlasgow, Scotland, GB
Permanent

Purview is seeking a skilled Python Developer to join their team in Glasgow, UK.This permanent role involves developing Athena, Quartz, and SecDB while ensuring a solid understanding of the core ar... Show more