Senior Software Engineer - Python_data Engineering

Lead Data Engineer

Perform statistical analysis, identify trends, patterns, and anomalies within datasets using Python libraries such as NumPy, SciPy, and Scikit-learn.

Develop and implement predictive models or machine learning algorithms to address specific business problems.

Looking for a lead engineer for data analytics with good knowledge in Python programming, experience in working with core data analytics libraries in Python (Pandas, NumPy). He/she should have experience with web frameworks & APIs (Flask, FastAPI) and containerization, Docker.

Experienced and having knowledge in building ETL pipelines, data engineering workflows, and data visualization (Spark, Plotly, Dash, SQL).

Experience in working with MS Azure Cloud and below Azure services (Databricks, Data Factory, Azure App Service, Azure Batch Service, CosmosDB, Azure Functions, Azure Datalake).

Machine learning concepts & libraries (time series forecasting, anomaly detection, failure prediction, scikit-learn, statsmodels, PyOD, TensorFlow, PyTorch).

Expert in Python programming language.

Strong understanding of Python syntax, data structures, OOP, and best practices.

Experience with writing efficient, maintainable, and scalable Python code.

Experience in working with core data analytics libraries in Python.

Pandas – Data manipulation and analysis.

NumPy – Numerical computations.

SciPy – Scientific computing, signal processing.

Scikit-learn – Classical machine learning algorithms.

Statsmodels – Statistical modeling and time series analysis.

Web frameworks & APIs.

Flask, FastAPI – RESTful APIs and web apps.

Experience and knowledge in building ETL pipelines and data engineering workflows, including:

Ability to work with structured and unstructured data from various sources.

Understanding of best practices for data quality, reliability, and scalability.

Proficiency in data ingestion, cleansing, transformation, and orchestration.

PySpark, Databricks – distributed data processing frameworks.

Azure Data factory -ETL tool.

Good SQL knowledge.

Data visualization and dashboards.

Plotly, Dash – Interactive visualizations and dashboards.

Matplotlib, Seaborn – Static visualizations.

Experience in working with MS Azure Cloud and below Azure services:

Azure Databricks.

Azure Data Factory.

Azure App Service.

Azure Batch Service.

Azure Cosmos DB.

Azure Datalake.

Azure DevOps.

Azure Functions.

Azure Container Registry.

Version control (Git).

Experience in containerization and Docker, including:

Building, deploying, and managing applications using Docker containers.

Writing and optimizing Docker files for Python and analytics projects.

Machine learning concepts & libraries.

Time series forecasting.

Failure prediction (classification/regression).

Feature engineering from sensor data.

Anomaly detection.

Model evaluation and deployment.

Scikit-learn, statsmodels, PyOD, TensorFlow, PyTorch.