Data Engineer - Assistant Vice President

Data Engineer - Assistant Vice President

We are looking for a hands-on data engineer who is passionate about solving business problems through innovation and engineering practices. As a data engineer, you will leverage your deep technical knowledge to drive the creation of high-quality software products. You will also be expected to share your technical expertise and promote a culture of technical excellence within the team. The data engineer will work with a team lead and will be a code-contributing member of the team that will deliver solutions against the sprint-level commitments.

Responsibilities

• Code contributing member of an agile team, working to deliver sprint goals.
• Demonstrating technical knowledge and expertise in software development, including programming languages, frameworks, and best practices.
• Actively contributes to the implementation of features and technical solutions. Write clean, efficient, and maintainable code that meets the highest standards of quality.
• Collaborate with other engineers to define and evolve the overall system architecture and design.
• Provide guidance on scalable, robust, and efficient solutions that align with business requirements and industry best practices.
• Offer expert engineering guidance and support to multiple teams, helping them overcome technical challenges, make informed decisions, and deliver high-quality software solutions. Foster a culture of technical excellence and continuous improvement.
• Stay up to date with emerging technologies, tools, and industry trends. Evaluate their potential impact on the organization and provide recommendations for technology adoption and innovation.

Required qualifications

• 3-5+ years' experience of implementing data-intensive solutions using agile methodologies.
• Proficient in one or more programming languages commonly used in data engineering such as Python, Java, or Scala.
• Multiple years of experience with software engineering best practices (unit testing, automation, design patterns, peer review, etc.)
• Experience with Hadoop for data storage and processing is valuable, as is exposure to modern data platforms such as Snowflake and Databricks.
• Multiple years of experience architecting and building horizontally scalable, highly available, highly resilient, and low latency applications
• Strong proficiency in working with relational databases and using SQL for data querying, transformation, and manipulation.
• Clear understanding of data structures and object-oriented principles.
• Experience of modeling data for analytical consumers
• Ability to automate and streamline the build, test, and deployment of data pipelines.
• Proven experience of providing technical vision and guidance to a data team.
• Thrives in a dynamic environment, capable of managing multiple tasks simultaneously while maintaining a high standard of work.
• BA/BS degree or equivalent work experience.

Preferred qualifications

• Multiple years of experience with cloud-native development and container orchestration tools (Serverless, Docker, Kubernetes, OpenShift, etc.).
• Familiarity with open-source data engineering tools and frameworks (e.g. Spark, Kafka, Beam, Flink, Trino, Airflow, DBT) is a valuable asset
• Exposure to a range of table and file formats including Iceberg, Hive, Avro, Parquet, and JSON
• Experience with data transformation tools like Ab Initio a plus. Appreciation of data parallelism, associated partitioning strategies, and an understanding of key components in the GDE is particularly relevant.
• Experience in cloud native technologies and patterns (AWS, Google Cloud)
• Exposure to infrastructure as code tools (i.e., Terraform, Cloudformation, etc.)
• Experience of driving and/or influencing the data strategy of your team or organization

Education

• BA/BS degree or equivalent work experience.