Azure Data Engineer(lead)

Azure Databricks Python Developer

Company Name:

Job Title: Azure Databricks Python Developer

Qualification: Any graduation

Experience: 10 to 15 years

Must Have Skills:

• Proficiency in writing clean, efficient, and production-ready Python code.
• Azure Databricks (Hands-on Experience)
• Experience building notebooks, jobs, and clusters using Azure Databricks.
• PySpark / Spark (Data Processing)
• Strong command over PySpark for processing big data on distributed systems.
• Azure Ecosystem Knowledge
• Experience with Azure services like ADLS, ADF, Azure Synapse, Key Vault, etc.
• Data Pipeline Development (ETL/ELT)
• Proven ability to build, schedule, and monitor end-to-end data pipelines.

Good to Have Skills:

• Experience working with Delta Lake tables and time travel features.
• CI/CD Integration
• Knowledge of integrating Databricks with DevOps pipelines (e.g., Azure DevOps, GitHub Actions).
• Data Modeling and Warehousing
• Understanding of dimensional modeling, data vault, or modern data stack.
• MLflow / Machine Learning Integration
• Exposure to ML model training, tracking, or serving within Databricks.
• Performance Tuning
• Ability to optimize Spark jobs, cluster configuration, and resource management.

Roles and Responsibilities:

• Design & Develop Scalable Pipelines
• Build and maintain scalable, reusable ETL pipelines using Databricks and PySpark.
• Collaborate with Stakeholders
• Work with data scientists, analysts, and product teams to understand data needs.
• Optimize Data Workflows
• Tune performance of Spark jobs, minimize data latency, and manage costs.
• Deploy & Monitor Jobs
• Automate job scheduling, error handling, and monitoring using Databricks Jobs/API.
• Ensure Data Quality and Governance
• Implement data validation, lineage, and security best practices using Azure tools.

Location: Bangalore, Hyderabad, and Chennai

CTC Range: 30 to 35 LPA

Notice period: Immediate joiners

Shift Timings: General

Mode of Interview: Virtual

Mode of Work: Hybrid

Mode of Hire: Permanent