Lead Data Engineer
Lead Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Job Overview
We are looking for a Data Engineer who will be part of our Analytics Practice and will be expected to actively work in a multidisciplinary fast paced environment. This role requires a broad range of skills and the ability to step into different roles depending on the size and scope of the project; its primary responsibility is the acquisition transformation loading and processing of data from a multitude of disparate data sources including structured and unstructured data for advanced analytics and machine learning in a big data environment.
Responsibilities:
Engineer a modern data pipeline to collect organize and process data from disparate sources.
Performs data management tasks such as conduct data profiling assess data quality and write SQL queries to extract and integrate data
Develop efficient data collection systems and sound strategies for getting quality data from different sources
Consume and analyze data from the data pool to support inference prediction and recommendation of actionable insights to support business growth.
Design and develop ETL processes using tools and scripting. Troubleshoot and debug ETL processes. Performance tuning and opitimization of the ETL processes.
Provide support to new of existing applications while recommending best practices and leading projects to implement new functionality.
Collaborate in design reviews and code reviews to ensure standards are met. Recommend new standards for visualizations.
Learn and develop new ETL techniques as required to keep up with the contemporary technologies.
Reviews the solution requirements and architecture to ensure selection of appropriate technology efficient use of resources and integration of multiple systems and technology.
Support presentations to Customers and Partners
Advising on new technology trends and possible adoption to maintain competitive advantage
Experience Needed:
8 years of related experience is required.
A BS or Masters degree in Computer Science or related technical discipline is required
ETL experience with data integration to support data marts extracts and reporting
Experience connecting to varied data sources
Excellent SQL coding experience with performance optimization for data queries.
Understands different data models like normalized denormalied stars and snowflake models. Worked with transactional temporarl time series and structured and unstructured data.
Experience on Azure Data Factory and Azure Synapse Analytics
Worked in big data environments cloud data stores different RDBMS and OLAP solutions.
Experience in cloudbased ETL development processes.
Experience in deployment and maintenance of ETL Jobs.
Is familiar with the principles and practices involved in development and maintenance of software solutions and architectures and in service delivery.
Has strong technical background and remains evergreen with technology and industry developments.
At least 3 years of demonstrated success in software engineering release engineering and/or configuration management.
Highly skilled in scripting languages like PowerShell.
Substantial experience in the implementation and exectuion fo CI/CD processes.
Additional Requirements
Demonstrated ability to have successfully completed multiple complex technical projects
Prior experience with application delivery using an Onshore/Offshore model
Experience with business processes across multiple Master data domains in a services based company
Demonstrates a rational and organized approach to the tasks undertaken and an awareness of the need to achieve quality.
Demonstrates high standards of professional behavior in dealings with clients colleagues and staff.
Is able to make sound and far reaching decisions alone on major issues and to take full responsibility for them on a technical basis.
Strong written communication skills. Is effective and persuasive in both written and oral communication.
Experience with gathering end user requirements and writing technical documentation
Time management and multitasking skills to effectively meet deadlines under timetomarket pressure
Employment Type
Full Time
