Total Exp: 7 Years
Detailed JD:
ETL/ELT Development: Design develop and manage data ingestion and transformation pipelines using Azure Data Factory (ADF) to extract load and transform data from various sources into our data lake and data warehouse.
- Data Processing: Utilize Azure Databricks with Spark (PySpark Scala) to perform complex data transformations processing and analysis on large datasets.
- Data Warehousing: Implement and manage our data warehouse solution using Azure Synapse Analytics including dedicated SQL pools and serverless SQL pools.
- SQL Development: Write and optimize complex SQL queries stored procedures and views for data manipulation analysis and reporting.
- Database Management: Work with both relational (Azure SQL Database) and NoSQL (Azure Cosmos DB) databases to support diverse data storage needs.
- Collaboration: Work closely with data scientists business analysts and other engineering teams to understand data requirements and deliver reliable data solutions.
Mandatory Skills:Azure Data Engineer