Data Engineer

Data Engineer Opportunity

We are looking for a highly skilled Data Engineer to join our team on a contract basis in Atlanta, Georgia. This role focuses on optimizing data processes and infrastructure, ensuring efficient data management and performance. The ideal candidate will possess expertise in modern data engineering tools and technologies.

Responsibilities:

• Optimize data indexing and address fragmentation issues to enhance system performance.
• Develop and maintain data pipelines using ETL processes to ensure accurate data transformation and integration.
• Utilize Apache Spark for scalable data processing and analytics.
• Implement and manage big data solutions with Apache Hadoop.
• Design and deploy real-time data streaming frameworks using Apache Kafka.
• Collaborate with cross-functional teams to identify and resolve data-related challenges.
• Monitor and improve system performance by analyzing data usage and storage trends.
• Write efficient code in Python to support data engineering tasks.
• Document processes and maintain clear records of data workflows and optimizations.
• Ensure data security and compliance with organizational standards.

Requirements:

• Proficiency in Apache Spark for large-scale data processing.
• Strong programming skills in Python, particularly for data handling and scripting.
• Hands-on experience with Apache Hadoop for big data storage and management.
• Knowledge of Apache Kafka for real-time data streaming.
• Expertise in ETL processes, including data extraction, transformation, and loading.
• Ability to diagnose and resolve data optimization and fragmentation issues.
• Familiarity with data security protocols and compliance standards.
• Effective communication skills to collaborate across teams and document workflows.