Senior Lead Data Engineer R
Share
Job Location:
Monthly Salary:
Posted on:
18 hours ago
Vacancies:
1 Vacancy
Job Summary
Senior Lead Data Engineer
Primary Skills
- AKS Event Hub Azure DevOps Cosmos DB Azure Functions
Specialization
- Azure Data Engineering Advanced: Senior Data Engineer
Job requirements
- Key Responsibilities:- ETL Pipeline Monitoring & Support oMonitor and manage multiple ETL pipelines supporting Novacore data platforms. oProvide end-to-end troubleshooting debugging and resolution of pipeline failures. oPerform root cause analysis for issues and implement permanent fixes. oProactively communicate pipeline failures resolutions and status updates to internal stakeholders and clients. Development & Architecture Design oDesign and develop new ETL pipelines and data processing workflows as per business requirements. oAssist clients in defining data architecture and proposing optimal technical solutions. oRecommend and implement scalable efficient and cost-effective ETL and data architecture approaches. oSupport enhancements and optimizations to existing pipelines and data models. Deployment & DevOps Support oManage code deployments to Production environments following established release processes. oUtilize Azure DevOps for source control CI/CD pipelines release management and work item tracking. oEnsure deployment best practices version control and rollback strategies are followed. Support Documentation & Collaboration oProvide ongoing operational support and maintenance for data pipelines and related systems. oCreate and maintain technical documentation including pipeline workflows architecture diagrams and runbooks. oCollaborate with cross-functional teams (engineering analytics and business stakeholders) to ensure smooth delivery and alignment with business goals. oAdhere to agreed SLAs timelines and quality standards.
We may use artificial intelligence (AI) tools to support parts of the hiring process such as reviewing applications analyzing resumes or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed please contact us.
Required Experience:
Senior IC
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
Brillio is a global leader in Enterprise Digital Transformation Solutions, providing strategic consulting services and solutions using emerging technologies.
