Senior Devops Databricks Engineer - Remote Eligible

Databricks Senior DevOps Engineer

We are looking for a Databricks Senior DevOps Engineer to design, build, and operate enterprise-grade data and ML platforms on AWS and Databricks for our Financial Crime platform.

This role focuses on platform infrastructure, governance, security, and operations, not pure data engineering or Spark coding. You will have full administrative ownership of the Databricks environment to ensure it is scalable, secure, compliant, and production-ready.

Key Responsibilities

• Architect, build, and operate end-to-end data and ML platforms on AWS and Databricks.
• Own and administer Databricks workspaces for the Financial Crime platform.
• Lead DevSecOps and DataOps practices, including infrastructure-as-code (IaC) and CI/CD pipelines for data and ML workflows.
• Configure and optimize Databricks compute clusters (job clusters and all-purpose clusters) for performance, scalability, and cost efficiency.
• Manage and enforce governance through Unity Catalog, including access control, security policies, data lineage, and isolation.
• Build and operate ML infrastructure, including model deployment and serving endpoints.
• Integrate AWS services (e.g., S3, Redshift, Kinesis, Lambda, EKS/ECS) with Databricks runtime and Delta Lake.
• Implement platform security best practices, including secrets management, audit logging, and compliance controls.
• Optimize system performance and diagnose large-scale production issues.
• Mentor engineering teams and define architectural best practices for high-scale data and ML systems.