Enter a job title or keyword
Engineer IT
Job Location:
Monthly Salary:
Posted:
12 July 2026 (30+ days ago)
Posted:
Application Deadline:
24 November 2026
Application Deadline:
Vacancies:
1 Vacancy
Vacancies:
Job Summary
Job Description:
Key Responsibilities
- Design develop and maintain scalable data pipelines using Databricks and PySpark
- Build robust ETL/ELT workflows for batch and near realtime data processing
- Write optimized and efficient SQL queries for data transformation validation and analysis
- Work with large-scale distributed datasets using Spark-based processing frameworks
- Perform data ingestion from multiple sources (APIs files databases) into data lake/lakehouse architecture
- Ensure data quality integrity and consistency through validations and monitoring
- Optimize pipeline performance and cost (partitioning caching query tuning etc.)
- Collaborate with cross-functional teams (Data Ops Business Analytics) to deliver data solutions
- Troubleshoot pipeline failures and implement proactive monitoring and alerting
- Maintain proper documentation for data models transformations and workflows
Required Skills & Experience
- Strong hands-on experience in:
- Databricks (Lakehouse platform notebooks jobs clusters)
- PySpark / Apache Spark
- Advanced SQL (joins window functions performance tuning)
- Solid understanding of ETL concepts data warehousing and data modeling
- Experience building scalable data pipelines for structured and semi-structured data
- Familiarity with data lake / delta lake architecture
- Good understanding of performance optimization techniques in Spark and SQL
- Experience with version control (Git) and deployment practices
- Strong problem-solving and analytical skills
Good to Have
- Experience with cloud platforms (AWS preferably)
- Exposure to CI/CD pipelines (Jenkins)
- Knowledge of Delta Lake Unity Catalog or similar governance tools
- Understanding of data quality frameworks / monitoring tools
- Experience working in Agile / Scrum teams
Location:
This position can be based in any of the following locations:
ChennaiCurrent Guardian Colleagues: Please apply through the internal Jobs Hub in Workday
Required Experience:
IC
About Company
We provide life insurance, disability insurance, dental insurance, and other benefits that help protect people and inspire their well-being.
