Lead Data Engineer - R
Lead Data Engineer - R
Posted on :
22-07-2025
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
About Brillio:
Brillio is one of the fastest growing digital technology service providers and a partner of choice for many Fortune 1000 companies seeking to turn disruption into a competitive advantage through innovative digital renowned for its world-class professionals referred to as Brillians distinguishes itself through their capacity to seamlessly integrate cutting-edge digital and design thinking skills with an unwavering dedication to client satisfaction.
Brillio takes pride in its status as an employer of choice consistently attracting the most exceptional and talented individuals due to its unwavering emphasis on contemporary groundbreaking technologies and exclusive digital projects. Brillios relentless commitment to providing an exceptional experience to its Brillians and nurturing their full potential consistently garners them the Great Place to Work certification year after year.
Lead Data Engineer
Primary Skills
- IICS Alation Data Modelling Fundamentals Data Warehousing ETL Fundamentals Modern Data Platform Fundamentals PLSQL T-SQL Stored Procedures Python SQL SQL (Basic Advanced) Talend
Job requirements
- Job Title: Lead Data Engineer
- We are seeking a highly skilled and experienced Senior Data Engineer to lead the development migration and optimization of data systems across our enterprise infrastructure.
- This role will play a critical part in migrating legacy data systems to Google BigQuery designing robust ETL/data pipelines and enabling real-time IoT data integration.
- The ideal candidate will have a strong background in cloud-based data warehousing particularly in Google Cloud Platform (GCP) and deep expertise in Python SQL Spark and data engineering best practices.
- 1. Data Migration & Cloud Modernization Analyze legacy on-premises and cloud data warehouse systems (e.g. SQL Server). Lead migration of large-scale datasets to Google BigQuery. Design and execute data migration strategies ensuring data quality consistency and performance.
- 2. Data Integration & Streaming Integrate data from diverse sources including APIs relational databases and unstructured data. Implement real-time streaming pipelines for IoT data ingestion and processing in large-scale environments.
- 3. ETL / Data Pipeline Development Evaluate and modernize legacy SSIS packages to cloud-native ETL pipelines. Design and develop scalable ETL workflows using Apache Airflow Python Spark and other GCP-native tools. Ensure data is processed transformed and loaded reliably into BigQuery for downstream analytics.
- 4. Programming & Query Optimization Write complex SQL queries stored procedures and scheduled queries in BigQuery. Build reusable modular data transformation scripts using Python Java Spark and SQL. Optimize query performance and manage cost efficiency in a cloud data warehouse environment.
- 5 years of experience in Data Engineering including at least 2 years working with Google Cloud Platform (GCP) and BigQuery.
- Strong experience with data migration from on-prem to cloud platforms. Proficiency in building ETL pipelines with Apache Airflow Python and Spark.
- Hands-on experience with streaming data ingestion (especially IoT data).
- Strong SQL development skills; experience with performance tuning in BigQuery.
- Solid understanding of cloud architecture and data warehouse design.
- Experience with version control systems (e.g. Git) and CI/CD for data pipelines.
- GCP Professional Data Engineer certification.
- Experience with tools like dbt Terraform or Kafka.
- Background in analytics machine learning pipelines or DevOps for data.
- #LI-SR1
Job Overview:
Key Responsibilities:
Required Skills & Experience:
Preferred Qualifications:
Know more about:
Brillio is an equal opportunity employer to all regardless of age ancestry colour disability (mental and physical) exercising the right to family care and medical leave gender gender expression gender identity genetic information marital status medical condition military or veteran status national origin political affiliation race religious creed sex (includes pregnancy childbirth breastfeeding and related medical conditions) and sexual orientation.
$80 - $85 an hour
Know what its like to work and grow at Brillio: Click here
Employment Type
Full-Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
