Big Data Engineer

Senior Software Engineer

We are seeking an experienced Senior Software Engineer with deep expertise in Spark, Scala, and Python to lead the implementation and optimization of large-scale financial models. This role involves translating complex mathematical equations into highly efficient, scalable code capable of processing massive datasets, ensuring models run at peak performance. The ideal candidate will have a strong background in distributed computing, performance tuning, and data transformation at scale.

Skills and Requirements:

• 5+ years experience in high performance Python development
• Heavy data background
• 3+ years Spark – version 3.3 ideally
• 1+ years Scala
• 1+ years Spark SQL
• Proven leadership abilities, with experience leading cross-functional teams or projects
• Ability to analyze large data sets – 10 billion rows, 500 columns

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.