Lead Data Engineer

Lead Data Engineer

We are seeking a highly skilled and motivated Data Engineer to join our innovative team. As a Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure to support our data-driven initiatives. You will collaborate closely with cross-functional teams to ensure the availability, reliability, and performance of our data systems and solutions.

Data Pipeline Development

• Design, implement, and optimize end-to-end data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data.
• Develop robust ETL (Extract, Transform, Load) processes to integrate data from diverse sources into our data ecosystem.
• Implement data validation and quality checks to ensure accuracy and consistency.

Data Modeling and Architecture

• Design and maintain data models, schemas, and database structures to support analytical and operational use cases.
• Optimize data storage and retrieval mechanisms for performance and scalability.
• Evaluate and implement data storage solutions, including relational databases, NoSQL databases, data lakes, and cloud storage services.

Data Infrastructure Management

• Configure and manage data infrastructure components, including databases, data warehouses, data lakes, and distributed computing frameworks.
• Monitor system performance, troubleshoot issues, and implement optimizations to enhance reliability and efficiency.
• Implement data security controls and access management policies to protect sensitive information.

Collaboration and Documentation

• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver tailored solutions.
• Document technical designs, workflows, and best practices to facilitate knowledge sharing and maintain system documentation.
• Provide technical guidance and support to team members and stakeholders as needed.

Requirements:

• Proven experience in data engineering, software development, or related roles.
• Proficiency in programming languages commonly used in data engineering (e.g., Python, Java, Scala, etc.).
• Strong knowledge of database systems, data modeling techniques, and SQL proficiency.
• Proficiency with ETL tools commonly used in data engineering (e.g., SSIS, Databricks, Azure Data Factory).
• Experience with big data technologies and frameworks (e.g., Spark, Kafka, etc.).
• Familiarity with cloud platforms and services (e.g. Azure, Snowflake, AWS, etc.).
• Excellent problem-solving skills and attention to detail.
• Effective communication and collaboration skills in a team-oriented environment.
• Ability to adapt to evolving technologies and business requirements.

Benefits: We are looking for intellectually curious people, passionate about the bigger picture of how the technology industry is evolving, ready to ask difficult questions and deal with complicated scenarios! If you are creative and a problem solver, this is the place to be as will be supporting you to fast-forward your career! We enhance each employee's potential through personal development through a wide range of learning tools both formal and informal. LSEG is a leading global financial markets infrastructure and data provider. Our purpose is driving financial stability, empowering economies and enabling customers to create sustainable growth. Our values of Integrity, Partnership, Excellence and Change underpin our purpose and set the standard for everything we do, every day. They go to the heart of who we are and guide our decision making and everyday actions. Working with us means that you will be part of a dynamic organization of 25,000 people across 65 countries. However, we will value your individuality and enable you to bring your true self to work so you can help enrich our diverse workforce. You will be part of a collaborative and creative culture where we encourage new ideas and are committed to sustainability across our global business. You will experience the critical role we have in helping to re-engineer the financial ecosystem to support and drive sustainable economic growth. Together, we are aiming to achieve this growth by accelerating the just transition to net zero, enabling growth of the green economy and creating inclusive economic opportunity. LSEG offers a range of tailored benefits and support, including healthcare, retirement planning, paid volunteering days and wellbeing initiatives. We are proud to be an equal opportunities employer. This means that we do not discriminate on the basis of anyone's race, religion, color, national origin, gender, sexual orientation, gender identity, gender expression, age, marital status, veteran status, pregnancy or disability, or any other basis protected under applicable law. Conforming with applicable law, we can reasonably accommodate applicants' and employees' religious practices and beliefs, as well as mental health or physical disability needs.