AI Productivity Engineer Intern

AI Productivity Engineer Intern

Yousign is looking for an AI Productivity Engineer Intern to join their AI Productivity team, directly under the supervision of the AI Productivity Lead Engineer. During this six-month internship, you will participate in the implementation of a robust and industrialized evaluation process for our AI productivity agents (Dust). Your scope will cover:

- Defining quality criteria

- Creating test sets

- Designing, developing, deploying, and continuously operating an evaluation pipeline.

The goal is to improve the quality, reliability, and mesurability of the agents used by the teams, thanks to followed performance indicators.

Main Missions:

• Benchmark & Study: Review of evaluation approaches (LLM-as-Judge, golden sets, pairwise, human-in-the-loop); benchmarking of tools (Dust, LangSmith, OpenAI Evals, custom platform); definition of quality criteria by type of agent (accuracy, robustness, security, cost, latency).
• Design: Design of a reproducible evaluation pipeline (datasets, orchestration, storage of runs, traceability); specification of integration with Dust agents and CI/CD; schema of dashboards and alerts (adoption, regressions, quality SLO).
• Development: Implementation of an evaluation platform (Python/TypeScript) and first test sets; LLM-as-Judge integration and metrics (accuracy, relevance, safety, cost, latency); automation of weekly campaigns and versioning of results.
• Deployment: Deployment of the pipeline, dashboards, and alerts in the internal production environment; integration into team workflows (documentation, training, runbooks); handover for continuous operation.

Profile Sought:

• Final year student at a French engineering school looking for a final year internship.

Technical Skills (Hard Skills):

Must-have:

• Solid foundations in development (Python or TypeScript) and metrics.
• Rigorous data management (datasets, annotation, reproducibility, versioning).
• Strong interest in generative AI and quality evaluation.
• Fluent English (technical documentation, tools).

Nice-to-have:

• Experience with LLMs (Claude, GPT, etc.) and automated evaluation.
• Knowledge of evaluation tools (LangSmith, OpenAI Evals, Langfuse) or observability.
• Practice of CI/CD, dashboards, and alerting.

Personal Qualities (Soft Skills):

• Curiosity and technological vigilance – Interest in AI innovation.
• Pedagogy – Ability to popularize and train.
• Adaptability – Rapidly evolving environment.
• Initiative spirit – Proactivity in identifying opportunities.

What You’ll Find at Yousign:

• A dynamic scale-up that puts people first
• A management style that promotes autonomy and cohesion through common objectives (OKR)
• The best of culture and working methods
• A supportive and stimulating work environment
• Regular team events
• The possibility to work from our beautiful offices in Paris near the Place de Clichy
• Occasional remote work (rhythm to be adapted with your team)

Equal Opportunity at Yousign

At Yousign, we celebrate uniqueness and ensure that every team member has the same opportunities to contribute. We offer equal chances to all and maintain transparent and fair processes.

Join us at Yousign, where our differences fuel creativity and strength — let’s embark on this incredible adventure together!