Overview
The Data Engineer plays a crucial role in the management and optimization of data systems within an organization. This position is vital for transforming raw data into a meaningful format that can be utilized by data scientists and analysts for generating insights. The Data Engineer collaborates closely with multiple stakeholders including data architects data scientists and software engineers to ensure that data infrastructure is reliable scalable and secure. They design and build systems that facilitate data collection storage processing and analysis. A deep understanding of data technologies and tools combined with the ability to analyze complex datasets makes the Data Engineer a key figure in enhancing data-driven decision-making processes. As organizations increasingly rely on data for strategy and performance improvements the demand for skilled data engineers continues to grow.
Key Responsibilities
- Design & build real-time data pipelines and analytics solutions.
- Implement Big Data/Data Lake projects in Azure Snowflake.
- Write and maintain clean modular and efficient code in Python and SQL.
- Handle distributed platform architecture and optimize large-scale data integrations.
- Follow Agile best practices with strong problem-solving capabilities.
Must-Have Skills:
- 4 years of experience with Azure Snowflake Databricks Kafka.
- Proficiency in Python complex SQL and Unix scripting.
- Strong knowledge of distributed systems and data lake implementations.
- Excellent communication multitasking and analytical skills.
- Agile methodology exposure and a self-starter mindset.
Nice to Have:
- Knowledge of Power BI Business Objects and Financial domain is a plus.
agile methodology,etl,data lake implementations,distributed systems,snowflake,databricks,unix scripting,python,kafka,azure,sql