Freelance Agent Evaluation Engineer

23 ore fa

Roma, Lazio, Italia Mindrift A tempo pieno

Please submit your CV in English and indicate your level of English proficiency.

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.

What this opportunity involves

While each project involves unique tasks, contributors may:

  • Create structured test cases that simulate complex human workflows
  • Define gold-standard behavior and scoring logic to evaluate agent actions
  • Analyze agent logs, failure modes, and decision paths
  • Work with code repositories and test frameworks to validate your scenarios
  • Iterate on prompts, instructions, and test cases to improve clarity and difficulty
  • Ensure that scenarios are production-ready, easy to run, and reusable

What we look for

This opportunity is a good fit for software engineers, open to part-time, non-permanent projects. Ideally, contributors will have: 

  • 3+ of software development experience with strong Python focus
  • Experience with Git and code repositories
  • Comfortable with structured formats like JSON/YAML for scenario description
  • Understanding core LLM limitations (hallucinations, bias, context limits) and how these affect evaluation design
  • Familiarity with Docker
  • English proficiency - B2

How it works

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid

Project time expectations

Tasks for this project are estimated to take 6-10 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.

Payment

  • Paid contributions, with rates up to $30/hour*
  • Fixed project rate or individual rates, depending on the project
  • Some projects include incentive payments

*Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

  • AI Agent Evaluation Analyst

    3 giorni fa

    Roma, Lazio, Italia Mindrift A tempo pieno

    This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English proficiency.At Mindrift, innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of...

  • Evaluation Scenario Writer

    2 settimane fa

    Roma, Lazio, Italia Mindrift A tempo pieno

    This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English.At Mindrift, innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI. What we...

  • MCP & Tools Python Developer - Agent Evaluation Infrastructure

    4 giorni fa

    Roma, Lazio, Italia Mindrift A tempo pieno

    This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English proficiency.At Mindrift, innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of...

  • Freelance Economics Expert

    2 settimane fa

    Roma, Lazio, Italia Mindrift A tempo pieno

    This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English proficiency.At Mindrift, innovation meets opportunity. We believe in using the power of collective intelligence to ethically shape the future of AI.What...

  • IT & Network Support Engineer/Technician

    2 settimane fa

    Roma, Lazio, Italia Tech Domain A tempo pieno

    We are looking for reliable Freelance IT & Network Support Engineer/Technician (Level 1 & Level 2) to support on-site IT tasks on a contractual basis. This is not a full-time position. Tasks will be shared based on project requirements, and you may accept assignments according to your availability.This freelance contract does not bind either party to ongoing...

  • Freelance Financial Analyst

    2 settimane fa

    Roma, Lazio, Italia Mindrift A tempo pieno

    This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English proficiency.At Mindrift, innovation meets opportunity. We believe in using the power of collective intelligence to ethically shape the future of AI.What...

  • Systems Engineer II

    1 settimana fa

    Roma, Lazio, Italia STERIS A tempo pieno

    At STERIS, we help our Customers create a healthier and safer world by providing innovative healthcare and life science product and service solutions around the globe. Position Summary The Systems Engineer II within the Reprocessing business will be responsible for testing and implementing new features, as well as enhancing existing ones. This position...

  • AI Consultant Freelance

    4 giorni fa

    Roma, Lazio, Italia ISA Digital Consulting A tempo pieno

    Isa Digital Consultingè una società indipendente che opera da 30 anni nella consulenza, specializzata in ICT Strategy & Architecture, Digital Solution e Human Resources. Ci occupiamo di supportare le aziende nella trasformazione digitale e nel miglioramento della performance della direzione ICT in Italia, Europa, Medio Oriente e Africa.Siamo alla ricerca...

  • Satellite IVVQ RF ENGINEER

    4 giorni fa

    Roma, Lazio, Italia Thales A tempo pieno

    A Joint Venture between Thales (67%) and Leonardo (33%), Thales Alenia Space is a global space manufacturer delivering, for more than 40 years, high-tech solutions for telecommunications, navigation, Earth Observation, environmental management, exploration, science and orbital infrastructures. Thanks to our diversity of skills, talents and cultures, our...

  • expert eu project manager

    3 giorni fa

    Roma, Lazio, Italia Tia Formazione internazionale Aps A tempo pieno

    Position: EU Project Proposal Writer (Senior)Contract: Freelance / Collaboration AgreementWe are seeking an experiencedEU Project Proposal Writerwith aminimum of 3 years of demonstrable experiencein the design and drafting of successful project proposals under EU programmes (e.g., Erasmus+, Creative Europe, Horizon Europe, CERV, Interreg, LIFE).Main...