Athena Step Functions Spark - Pyspark ETL Fundamentals SQL (Basic Advanced) Glue Python Lambda Data Warehousing EBS /EFS AWS EC2 Lake Formation Aurora S3 Modern Data Platform Fundamentals PLSQL Data Modelling Fundamentals Cloud front
Specialization
AWS Data EngineerIng Basic: Senior Data Engineer
Job requirements
Job Description AWS Senior Data Engineer
Company: Brillio Technologies Role: Senior Data Engineer Experience: 510 Years Location: India (Hybrid/Remote) Employment Type: Full-time
Role Overview
We are looking for a Senior Data Engineer with strong Databricks and Big Data expertise to design build and optimize scalable data pipelines and platforms. The ideal candidate will have hands-on experience with Spark PySpark AWS ecosystem and modern data architectures along with exposure to APIs and real-time data processing.
Key Responsibilities
Design and build scalable data pipelines using Spark / PySpark on Databricks
Develop and optimize ETL/ELT workflows for large-scale data processing
Work with AWS EMR S3 and Hadoop ecosystem for distributed data processing
Build and maintain data lake and data warehouse solutions
Develop and integrate APIs for data ingestion and consumption
Implement data processing workflows using Airflow / Autosys
Optimize data performance using partitioning caching and query tuning
Handle structured and semi-structured data from multiple sources
Ensure data quality governance and reliability of pipelines
Collaborate with cross-functional teams (Analytics Product Engineering)
Technical Skills (Must-Have)
Strong expertise in:
Apache Spark / PySpark
Databricks (mandatory)
SQL (PostgreSQL or similar)
Hands-on experience with:
AWS EMR S3
Hadoop Hive ecosystem
Programming skills:
Python (must-have)
Scala (good exposure)
Strong knowledge of:
UNIX / Shell scripting
ETL pipelines and data engineering fundamentals
Additional Skills
Experience with:
Elasticsearch (data storage & retrieval)
Workflow orchestration tools (Airflow Autosys)
Exposure to:
API development & integration
Version control tools (Git / SVN)
Basic knowledge of HTML (for web/data tasks)
Experience Required
Minimum 5 years of Data Engineering experience
Strong hands-on experience in Databricks Big Data stack
Proven experience working on large-scale distributed systems
Education
Bachelors or Masters degree in Computer Science Engineering or related field
Why Join Brillio
Work on cloud-native large-scale data platforms
Exposure to cutting-edge Databricks & AWS ecosystems
Collaborative and high-performance engineering culture
We may use artificial intelligence (AI) tools to support parts of the hiring process such as reviewing applications analyzing resumes or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed please contact us.
Required Experience:
Senior IC
Lead Data EngineerPrimary SkillsAthena Step Functions Spark - Pyspark ETL Fundamentals SQL (Basic Advanced) Glue Python Lambda Data Warehousing EBS /EFS AWS EC2 Lake Formation Aurora S3 Modern Data Platform Fundamentals PLSQL Data Modelling Fundamentals Cloud frontSpecializationAWS Data EngineerIng...
Lead Data Engineer
Primary Skills
Athena Step Functions Spark - Pyspark ETL Fundamentals SQL (Basic Advanced) Glue Python Lambda Data Warehousing EBS /EFS AWS EC2 Lake Formation Aurora S3 Modern Data Platform Fundamentals PLSQL Data Modelling Fundamentals Cloud front
Specialization
AWS Data EngineerIng Basic: Senior Data Engineer
Job requirements
Job Description AWS Senior Data Engineer
Company: Brillio Technologies Role: Senior Data Engineer Experience: 510 Years Location: India (Hybrid/Remote) Employment Type: Full-time
Role Overview
We are looking for a Senior Data Engineer with strong Databricks and Big Data expertise to design build and optimize scalable data pipelines and platforms. The ideal candidate will have hands-on experience with Spark PySpark AWS ecosystem and modern data architectures along with exposure to APIs and real-time data processing.
Key Responsibilities
Design and build scalable data pipelines using Spark / PySpark on Databricks
Develop and optimize ETL/ELT workflows for large-scale data processing
Work with AWS EMR S3 and Hadoop ecosystem for distributed data processing
Build and maintain data lake and data warehouse solutions
Develop and integrate APIs for data ingestion and consumption
Implement data processing workflows using Airflow / Autosys
Optimize data performance using partitioning caching and query tuning
Handle structured and semi-structured data from multiple sources
Ensure data quality governance and reliability of pipelines
Collaborate with cross-functional teams (Analytics Product Engineering)
Technical Skills (Must-Have)
Strong expertise in:
Apache Spark / PySpark
Databricks (mandatory)
SQL (PostgreSQL or similar)
Hands-on experience with:
AWS EMR S3
Hadoop Hive ecosystem
Programming skills:
Python (must-have)
Scala (good exposure)
Strong knowledge of:
UNIX / Shell scripting
ETL pipelines and data engineering fundamentals
Additional Skills
Experience with:
Elasticsearch (data storage & retrieval)
Workflow orchestration tools (Airflow Autosys)
Exposure to:
API development & integration
Version control tools (Git / SVN)
Basic knowledge of HTML (for web/data tasks)
Experience Required
Minimum 5 years of Data Engineering experience
Strong hands-on experience in Databricks Big Data stack
Proven experience working on large-scale distributed systems
Education
Bachelors or Masters degree in Computer Science Engineering or related field
Why Join Brillio
Work on cloud-native large-scale data platforms
Exposure to cutting-edge Databricks & AWS ecosystems
Collaborative and high-performance engineering culture
We may use artificial intelligence (AI) tools to support parts of the hiring process such as reviewing applications analyzing resumes or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed please contact us.
Brillio is a global leader in Enterprise Digital Transformation Solutions, providing strategic consulting services and solutions using emerging technologies.
Job Location:
Monthly Salary:
Posted on:
Vacancies:
