Overview
The Data Engineer plays a vital role in handling data management and ensuring the accessibility of data for analytical and operational purposes within the organization. This position is crucial as data-driven decision-making increasingly guides business strategies. The Data Engineer is responsible for developing architecture and data processing systems working closely with data scientists and analysts to facilitate the efficient use and flow of data. This role involves designing and maintaining robust pipelines that can handle large volumes of data and they must be skilled in various programming languages and tools. As companies rely more on big data to drive decisions the responsibility of the Data Engineer has expanded requiring a competent individual who can ensure data quality manage data integrations and optimize data storage solutions. Individuals in this position are essential to transforming raw data into actionable insights that empower other teams to achieve their objectives.
Key Responsibilities
4 - 8 years of experience in Data Engineer role.
4 years of experience in designing & building real time data pipelines and analytical solutions using big data and cloud technology ecosystem (Azure Snowflake Databricks kafka etc)
Expertise with Data Lake/Big Data Projects implementation in cloud (preferably AzureSnowflake)
Strong hand-on experience of Snowflake keeping up to date with available latest features.
Strong understanding of how distributed platforms works.
Strong hands-on experience on python programming language.
Strong hands-on experience on writing complex SQLs.
Ability to develop maintain and distribute the code in modularized fashion.
Process oriented focused on standardization streamlining and implementation of best practices delivery approach
Basic understanding of Unix shell script and Unix OS platform
Excellent problem solving and analytical skill
Excellent verbal and written communication skills
Ability to multi-task and function efficiently in a fast-paced environment
Self-starter with flexibility and adaptability in a dynamic work environment
Good Understanding of Agile Methodology
Secondary Skills / Desired skills (Not Mandatory)
Experience in Data Reporting Tools (preferably Business Objects)
Experience in Visualization Tools (preferably Power BI)
Financial Domain experience will be a plus
sql,cloud technologies,data engineering,python,kafka,azure,agile methodology,data visualization,data warehousing,data modeling,snowflake,databricks,big data,cloud technology,data lake,unix,sql proficiency,api development