Associate Manager - Data Engineer

Data Pipeline Automation Role

In this role, you will play a key role in automating and streamlining the data processing pipeline for AMESA markets.

Responsibilities

• Own data pipeline development end-to-end, spanning data modeling, testing, scalability, operability and ongoing metrics.
• Ensure that we build high quality software by reviewing peer code check-ins
• Define best practices for product development, engineering, and coding as part of a world class engineering team in Hyderabad
• Focus on delivering high quality data pipelines and tools through careful analysis of system capabilities and feature requests, peer reviews, test automation, and collaboration with QA engineers
• Develop software in short iterations to quickly add business value
• Introduce new tools / practices to improve data and code quality; this includes researching / sourcing 3rd party tools and libraries, as well as developing tools in-house to improve workflow and quality for all data engineers

Qualifications

• 4-6 years experience developing data pipelines / ETLs
• Deep understanding of database design and engineering
• Strong familiarity with automated test frameworks
• Current skills in following technologies:
  • Python
  • Airflow, Luigi, or similar orchestration platforms
  • Relational databases – Postgres, MySQL, or similar
  • AWS, Azure, or similar cloud platforms
  • GitHub or similar source control
  • Automated build process and tools