Staff AI Engineer - Remote Eligible

Staff AI Engineer

Datavant is the data collaboration platform trusted for healthcare. Guided by our mission to make the world's health data secure, accessible and actionable, we provide critical data solutions for organizations across the healthcare ecosystem - including providers, health plans, researchers, and life sciences companies. From fulfilling a single patient's request for their medical records to powering the AI revolution in healthcare, Datavanters are building the future of how data is connected and used to improve health. By joining Datavant today, you're stepping onto a driven and highly collaborative team that is passionate about creating transformative change in healthcare.

The Role

You will build agent-driven applications and AI systems that combine model reasoning with deterministic systems and healthcare data, with a focus on reliability, evaluation, and production readiness. You will also design and deploy agentic systems, LLM harnesses, and evaluation workflows that integrate models, structured data, and production services into real-world applications.

What You'll Do

Agent Development

• Design and implement agentic workflows including tool use, retrieval, and multi-step reasoning
• Build AI-driven applications that integrate LLMs with APIs, data pipelines, and analytics systems
• Develop reusable abstractions for context management, tool use, and orchestration

Harness Engineering

• Design and build LLM harnesses that structure how agents operate in production
• Create frameworks for prompt management, context injection, execution loops, and agent control flow
• Standardize patterns for integrating models with internal systems and external APIs

Evaluation

• Use and extend evaluation frameworks to measure correctness, reliability, and safety
• Define metrics for model performance within real workflows
• Build datasets and test cases to validate agent and system behavior over time

Systems & Infrastructure

• Integrate AI capabilities into production backend services
• Design pipelines combining retrieval, structured data, and model reasoning
• Optimize latency, cost, and scalability of AI-powered systems

Reliability & Observability

• Implement logging, tracing, and monitoring for agent behavior and outputs
• Debug and improve multi-step system performance and failure modes

Collaboration

• Partner with Product, Data, and Science teams to build AI-driven solutions
• Contribute to AI platform and architectural decisions
• Mentor engineers on AI engineering practices

Ways of Working

• Practice compound engineering: treat each system as an opportunity to improve how future systems are built
• Work in iterative loops across humans and agents: build, evaluate, refine, and codify
• Capture learnings from failures, evaluations, and edge cases, and encode them into prompts, tools, and workflows
• Build reusable assets such as agent patterns, harness components, and evaluation utilities that improve over time
• Contribute to systems and engineering practices that become more reliable and easier to extend with each iteration

What You'll Bring

Required

• 5+ years of software engineering experience
• Experience building applications with LLMs or agent frameworks
• Experience integrating APIs, data systems, and model outputs
• Strong understanding of system design, testing, and production reliability

Preferred

• Experience building LLM harnesses or agent frameworks
• Experience using evaluation frameworks for LLM systems
• Familiarity with retrieval systems, vector databases, or RAG patterns
• Experience with cloud platforms such as AWS, GCP, or Azure
• Experience in healthcare or other regulated environments