✨ About The Role
- The role involves building model pretraining pipelines and developing benchmarks for AI data workloads.
- Collaboration with the Daft data engine team is key to optimizing the data engine for modern AI workloads.
- The candidate will stay at the forefront of AI research, incorporating the latest advancements into the data engine and platform capabilities.
- Responsibilities include implementing advanced dataset and model training techniques, such as multimodal learning and synthetic data generation.
- The position requires a principled and observable approach to building training and data pipelines.
⚡ Requirements
- The ideal candidate will have strong programming skills in Python and experience with deep learning frameworks such as PyTorch or TensorFlow.
- A deep understanding of transformer architectures, self-supervised learning, and AI model training techniques is essential for success in this role.
- Experience with distributed training frameworks and efficient model parallelism will be crucial for optimizing performance.
- The candidate should possess expertise in data pipelines and large-scale dataset management, particularly with massive datasets over 100TB.
- Familiarity with systems programming languages like Rust or C++ is a plus, indicating a well-rounded technical background.
- A PhD or equivalent research experience in Machine Learning, Computer Science, or related fields is preferred, showcasing a strong academic foundation.