Role - Azure Data Engineer
Experience - 5-8 Yrs
Location - Bangalore (Hybrid ) 2 Days work from office
About Role :
We are looking for a talented Azure Data Engineer to join our team. The ideal candidate will have a strong background in designing, implementing, and managing data solutions on the Azure platform. You will work closely with cross-functional teams to develop robust data pipelines, optimize data storage and processing, and ensure data quality and reliability.
Responsibilities :
- Design, develop, and deploy data pipelines and ETL processes on the Azure platform.
- Implement and optimize data storage solutions using Azure Data Lake Storage, Azure SQL Database, Azure Cosmos DB, etc.
- Develop and maintain scalable data processing solutions using Azure Databricks, Apache Spark, or other technologies.
- Collaborate with data scientists and analysts to deploy machine learning models and advanced analytics solutions.
- Ensure data quality and reliability by implementing data validation, monitoring, and governance processes.
- Troubleshoot and resolve issues related to data pipelines, infrastructure, and performance.
- Stay updated with the latest advancements in Azure data services and best practices in data engineering.
Requirements :
- Create Databricks Notebooks and Catalogues
- Data normalization and ingestion using ADF pipelines
- Data enrichment on bronze, Silver and Gold layers
- Advanced python knowledge
- Professional hand-on experience in scala/python
- Professional hand-on experience in SQL and Query Optimization
- Build the infrastructure required for optimal transformation and integration from a wide variety of data sources using appropriate data integration technologies.
- Data Normalization
- Azure Data Factory pipelines creation and management for Data Enrichment and ingestion
- Azure Databricks Catalogues creation
- SQL Warehouse creation and management
About Statusneo :