AWS Data Engineer
Virtusa
Senior Lead Software Engineer
We are looking for a skilled AWS Data Engineer with strong experience in building and managing cloud-based ETL pipelines using AWS Glue, Python/PySpark, and Athena, along with data warehousing expertise in Amazon Redshift. The ideal candidate will be responsible for designing, developing, and maintaining scalable data solutions in a cloud-native environment.
Design and implement ETL workflows using AWS Glue, Python, and PySpark.
Develop and optimize queries using Amazon Athena and Redshift.
Build scalable data pipelines to ingest, transform, and load data from various sources.
Ensure data quality, integrity, and security across AWS services.
Collaborate with data analysts, data scientists, and business stakeholders to deliver data solutions.
Monitor and troubleshoot ETL jobs and cloud infrastructure performance.
Automate data workflows and integrate with CI/CD pipelines.
Required Skills & Qualifications:
- Hands-on experience with AWS Glue, Athena, and Redshift.
- Strong programming skills in Python and PySpark.
- Experience with ETL design, implementation, and optimization.
- Familiarity with S3, Lambda, CloudWatch, and other AWS services.
- Understanding of data warehousing concepts and performance tuning in Redshift.
- Experience with schema design, partitioning, and query optimization in Athena.
- Proficiency in version control (Git) and agile development practices.