Databricks Engineer/ Architect - EST Timezone - Remote Eligible

Databricks Engineers / Architects

We are looking for experienced Databricks Engineers / Architects to join a fast-moving data transformation initiative. The client is already live on Databricks with production pipelines and reporting in place, but the platform requires stabilization, optimization, and scaling.

This is an opportunity to work in a high-impact environment, helping transform an evolving data platform into a reliable, production-grade system delivering consistent business value.

Key Responsibilities:

• Design, build, and stabilize Databricks-based data pipelines
• Perform pipeline optimization and production hardening
• Develop and support reporting and data visualization solutions
• Implement best practices for performance, scalability, and cost optimization
• Work on end-to-end data workflows (ingestion → transformation → reporting)
• Identify and resolve performance bottlenecks and system inefficiencies
• Bring structure and consistency to an evolving Databricks environment
• Collaborate with cross-functional teams to improve platform reliability and ROI

Required Skills & Experience:

• Strong hands-on experience with Databricks (must-have)
• Experience in data pipeline development (ETL/ELT)
• Solid understanding of data engineering and data architecture principles
• Experience working with large-scale data processing systems
• Exposure to reporting/BI tools (Power BI, Tableau, or similar)
• Ability to work in fast-paced, evolving environments with minimal handholding

Preferred / Good-to-Have Skills:

• Familiarity with Apache Spark (PySpark / Scala)
• Experience with cloud platforms (AWS / Azure / GCP)
• Knowledge of performance tuning and cost optimization in Databricks
• Exposure to data governance, security, and best practices
• Prior experience in stabilizing or scaling data platforms

Ideal Candidate Profile:

• Can quickly ramp up and contribute from day one
• Strong problem-solver with experience handling unstable or partially built systems
• Comfortable working in a "build while running" environment
• Balances technical expertise with delivery focus

What Success Looks Like:

• Stabilized and optimized Databricks pipelines
• Improved platform reliability and performance
• Scalable and production-ready data workflows
• Consistent and measurable business value delivery