View All Jobs 113542
Data Engineer
Own the end-to-end data migration ETL pipelines for a multi-system integration project
Pune, Mahārāshtra, India
Mid-Level
10 hours agoBe an early applicant
Accenture
Global professional services firm providing strategy, consulting, digital, technology, and operations services to enterprises and governments worldwide.
10 Similar Jobs at Accenture
Data Engineer
Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.
Must have skills: AWS Glue, PySpark
Minimum 5 years of experience is required. 15 years full time education required.
Key Responsibilities
- Implement end-to-end ETL pipelines using AWS native services like Step Function, EventBridge, Glue (PySpark, SQL scripts), and Lambda for data extraction, transformation, and loading.
- Use pre-created utility for seamless migration, handling high-volume datasets with error handling and retry mechanisms.
- Collaborate with the Solution Designer to test pipelines for performance, and deploy with help of devOps engineer.
- Monitor and troubleshoot pipelines using CloudWatch, optimize for cost and process scalability, and document code for team handover.
- Support data quality validation and incremental loads to maintain data integrity during the hydration process.
- Engage with multiple teams and contribute on key decisions. Provide solutions to problems for their immediate team and across multiple teams.
Professional & Technical Skills:
- Proficiency in AWS Glue, Pyspark, SQL, ETL, Unix, Iceberg, Astronomer, DW concepts
- Strong understanding of data pipeline architecture and design principles.
- Experience with data warehousing solutions and ETL processes.
- Familiarity with cloud computing services and data storage solutions.
- Ability to troubleshoot and optimize data workflows for performance.
This position is based at our Pune office.
About Accenture
Global professional services firm providing strategy, consulting, digital, technology, and operations services to enterprises and governments worldwide.