We are looking for an experienced application development lead with deep technical expertise to join State Street Data Intelligence team. The candidate will play a key role in designing, building and maintaining a robust and scalable data processing platform and end to end solutions using Apache Spark, Databricks, and other related big data and distributed computing technologies.
What You Will Be Responsible For
The candidate will be deeply involved in all phases of the software development lifecycle, including scope definition, requirements analysis, functional and technical design, application build, unit testing, production deployment and support.
Provide architectural direction across development teams and lead technical design reviews.
Participate in strategic planning, technical roadmap development, and system architecture reviews.
Ensure all technical solutions exhibit higher level of efficiency, performance, security, scalability, and reliability
Collaborate with cross-functional teams to define technical requirements and ensure successful project execution.
Leverage AI and machine learning tools to drive platform innovation
Stay current with industry best practices; evaluate emerging tools for strategic benefit.
What We Value
Be capable of working independently in all phases of the software development lifecycle, including scope definition, requirements analysis, functional and technical design, application build, unit testing, production deployment and support.
Proven track record designing and building cloud native enterprise scale data computing solutions
Strong leadership skills, analytical problem-solving skills, quick to learn and adapt.
Self-motivated, creative problem solver, organized, collaborative with excellent communication skills.
Education & Preferred Qualifications
BS/MS in Computer Science or equivalent field
3-5 years of software engineering experience
Strong programming skills in Java, Python, Scala, SQL, UNIX and Shell scripting
Deep understanding of distributed system concepts like concurrency, fault tolerance, and consistency models.
Deep understanding of cloud-native and system integration design patterns, with the ability to apply patterns to build scalable, maintainable, and secure cloud applications
In-depth knowledge of event-driven architecture and real-time data streaming such as Kafka, AWS SQS, Kinesis,
Solid understanding of data security principles including encryption, key management, and secure design principles
Strong understanding of application monitoring, observability, and alerting practices to ensure system reliability, performance, and rapid incident response
Strong experience with batch and real-time data processing, focusing on data quality.
Strong experience with OLAP and OLTP Databases and storage solutions
Strong experience with Apache Spark, Databricks
Hands-on experience with MATLAB and MCR
Hands-on experience with system and application performance optimization
Hands-on experience with CICD and Dev. Ops.