Senior Data Engineer Contract

• Design build and maintain data pipelines in the Azure environment ensuring smooth operation in both batch and realtime processing.
• Optimize Azure data infrastructure to facilitate accurate data extraction transformation and loading from a variety of data sources.
• Develop and automate ETL processes using tools like Pyspark Azure Data Factory and Databricks to facilitate data extraction and manipulation from diverse sources.
• Transform raw data stored in Azure Data Warehouses into accessible datasets suitable for both technical and nontechnical stakeholders.
• Demonstrate expertise in Azure services and big data technologies including but not limited to Azure Data Factory Databricks and other relevant tools. Proficiency in SQL and Python is also crucial.
• Engage in business discussions collaborate with stakeholders and take ownership of deliverables ensuring that the data engineering solutions align with business objectives.
• Lead development teams or project teams providing guidance and expertise to ensure the successful execution of data engineering projects.
• Function as an independent developer who can guide design discussions and make informed decisions to drive project success.
• Leverage extensive experience to handle larger projects and manage multiple teams effectively.
• Exhibit excellent communication skills translating functional and technical requirements into actionable solutions that meet business needs.

• Minimum of 4 years of handson experience in Azure data engineering.
• Proficiency in a range of key technologies is essential including Pyspark Azure Data Factory (ADF) Databricks big data tools SQL and Python.
• Should be adept at handling business discussions and be capable of taking ownership of deliverables.
• Effective stakeholder management is crucial including leading development teams or project teams to ensure successful project outcomes.
• Selfsufficient developer who can provide guidance in design discussions and make informed decisions.
• Demonstrated experience in successfully managing larger projects and multiple teams is a key requirement.
• Excellent communication skills are necessary to understand and bridge the gap between functional and technical requirements providing practical and effective solutions.

azure,databricks,azure data factory (adf),big data technologies,pyspark,sql,azure data engineering,azure data factory,python