Data Engineer, Companionproduct&servi(compas)

Data Engineer For Ai Initiatives

The Companion Products and Services BI team is seeking an exceptional Data Engineer for AI initiatives who is highly data-driven and committed to driving business success. You possess demonstrated expertise in a range of data engineering technologies and a passion for continuously learning new tools and techniques. Rather than being narrowly tool-centric, you astutely determine the most effective technologies to address the task at hand. Importantly, you also excel at translating complex concepts into simple, easy-to-understand terms for your non-technical customers.

As a Data Engineer in this role, you will operate within one of the world's largest and most sophisticated data warehouse environments. Your responsibilities will include designing, implementing, and supporting scalable data infrastructure solutions. This will involve seamlessly integrating data from a diverse array of heterogeneous sources, aggregating and retrieving data quickly and securely, and curating high-quality datasets to power reporting, analysis, machine learning models, and ad-hoc data requests, plus you will also own and maintain our very own Redshift clusters to house this data. You will have the opportunity to work extensively with cutting-edge cloud data technologies.

Excellent business acumen and communication skills are essential, as you will collaborate closely with business stakeholders and technical leaders. Together, you will gather infrastructure requirements, architect data infrastructure, build data pipelines and datasets, and deliver solutions that meet evolving business needs. Importantly, you also maintain a keen awareness of emerging industry trends, proactively investigating and implementing new technologies where appropriate.

This dynamic role calls for a seasoned Data Engineer who thrives in complex, fast-paced environments. If you possess the blend of technical expertise, business savvy, and the ability to translate complex technical concepts into clear and actionable insights for our business stakeholders, we invite you to join our innovative team in shaping the future of Companion's data-driven capabilities.

Key job responsibilities:

• Use AWS tool kit for DE related automation and AI/ML production systems.
• Design and develop the pipelines required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Python and AWS big data technologies.
• Oversee and continually improve production operations, including optimizing data delivery, re-designing infrastructure for greater scalability, code deployments, bug fixes and overall release management and coordination.
• Establish and maintain best practices for the design, development and support of data integration solutions, including documentation.
• Work closely with Product teams, Data Scientists, Software developers and Business Intelligence Engineer to explore new data sources and deliver the data.
• Able to read, write, and debug data processing and orchestration code written Python/Scala etc following best coding standards (e.g. version controlled, code reviewed, etc.)