Data Engineer

Data Engineer Role

Strong technical, analytical, and problem-solving skills

Strong organizational skills, with the ability to work autonomously as well as in a team-based environment

Data pipeline template development

Data pipeline Monitoring development & support (operations)

The candidate must demonstrate proficiency in:

On-premise servers for data processing and extraction

Ability to own and deliver on large, multi-faceted projects

Fluency in complex SQL and experience with RDBMSs

(Bigdata, Spark, PySpark, Scala, Python, NiFi, Hive, NoSql DBs)

Experience designing and building big data pipelines

Experience working on large scale, distributed systems

Experience working on Azure Cloud services like (Azure Data Factory, Databricks, ADLS, Azure HD Insights, Azure Synapse, Azure Data warehouse)

Strong hands-on experience of programming language like PySpark, Scala with Spark, Python.

Exposure to various ETL and Business Intelligence tools

Solid grounding in Agile methodologies

Experience with git and other source control systems

Strong communication and presentation skills

Nice-to-have skills:

Certification in Hadoop/Big Data – Hortonworks/Cloudera

Azure Cloud certification

Unix or Shell scripting

Strong delivery background across the delivery of high-value, business-facing technical projects in major organizations

Experience of managing client delivery teams, ideally coming from a Data Engineering / Data Science environment