Senior Data Engineer - Ai - accelerated Data Architecture & Platforms - Remote Eligible

Senior Data Engineer

Whiteshield is a global policy advisory and AI economics firm known for our ability to respond to global challenges rapidly and incisively. We use the most advanced tools and technology and combine them with our team of leading international experts to engage decision-makers in tackling society's most significant challenges. We are recognized for our rapid decision support, innovation, data science algorithms and deep policy expertise. We are specialists in "connecting the dots" between policy, business and technology.

Whiteshield X is Whiteshield's product business, hosting our PaaS and SaaS product offering. Partnered with Google Cloud, we bring advanced solutions to governments around the world to drastically improve citizen outcomes in areas such as employment, trade, and education. We develop a host of different SaaS modules from job matching to managing scholarship programs, and from delivering financial support to growth support for SMEs. These modules support governments around the world with delivering G2C and G2B services efficiently and significantly improve policy outcomes.

We are seeking a Senior Data Engineer who combines deep technical expertise with architectural vision to build the next generation of AI-accelerated data platforms.

You will play a pivotal role in defining and implementing the organization's data architecture and platform strategy — ensuring that data is reliable, well-structured, and easily consumable across systems and applications. This is both a strategic and hands-on position: you'll design scalable architectures, build data pipelines, and collaborate with application and product teams to embed data excellence into every part of the technology stack.

If you're passionate about data craftsmanship, AI-driven automation, and modern cloud-native ecosystems, this is an opportunity to make a large-scale impact.

What You'll Do

• Data Architecture & Strategy
• Define and maintain the enterprise data architecture blueprint, spanning ingestion, lake, warehouse, and semantic layers.
• Collaborate closely with application and platform teams to design robust data strategies — ensuring alignment between application design, API contracts, and long-term data scalability.
• Develop integration patterns and governance models that support cross-platform interoperability, data lineage, and secure data sharing.
• Partner with cloud architects to ensure performance, scalability, and compliance across Azure environments.
• Data Engineering & Orchestration
• Build and optimize end-to-end data pipelines using Apache Airflow, Azure Fabric, Azure Data Factory, and SSIS — integrating diverse data sources (transactional, streaming, and analytical).
• Implement AI-accelerated data processing for intelligent schema mapping, deduplication, and performance optimization.
• Develop and maintain ETL/ELT frameworks using SQL and Python (Pandas, PySpark) for high-volume, low-latency data movement.
• Data Modeling & Warehousing
• Design semantic data models, dimensional schemas, and data marts using SSAS, PostgreSQL, MSSQL, and Azure Synapse/Fabric.
• Collaborate with BI and analytics teams to support dashboards, reports, and forecasting tools with clean, contextualized, and well-documented data models.
• Performance, Quality & Governance
• Define standards for data quality, lineage, observability, and access control.
• Implement automated validation, monitoring, and reconciliation mechanisms.
• Work with governance and security teams to ensure compliance with national and organizational data frameworks.
• Collaboration & Enablement
• Partner with application engineers, data scientists, and business analysts to deliver AI-ready, high-quality data products.
• Act as a bridge between application development and data architecture, ensuring consistent data design across services.
• Mentor junior data engineers and champion best practices in modern data engineering and DevOps for data.