Sr Data Engineer + PySpark, Python
Share
Job Location:
Monthly Salary:
Posted on:
Yesterday
Vacancies:
1 Vacancy
Job Summary
We are seeking a highly experienced and motivated Senior Data Engineer to join our team. The ideal candidate will have extensive experience in designing building and optimizing highly scalable and robust ETL/ELT pipelines. This role will be critical in shaping our data architecture implementing Lakehouse solutions and working with cutting-edge technologies like LLMs and Vector Search.
Key Responsibilities:
Design develop and maintain robust and scalable data pipelines using PySpark and Python.
Implement and manage data solutions within the Databricks platform.
Define and enforce data modeling standards specifically utilizing the Medallion architecture (Bronze Silver Gold layers).
Architect and implement Lakehouse capabilities including AI/Machine Learning features and Vector Search for advanced data retrieval.
Evaluate integrate and work with Large Language Model (LLM) frameworks.
Collaborate with data scientists and business stakeholders to understand data requirements and translate them into technical solutions.
Ensure data quality reliability and security throughout the data lifecycle.
Mentor junior engineers and contribute to best practices in data engineering.
Required Experience:
Senior IC
About Company
At Virtusa, we are builders, makers, and doers. Digital engineering is in our DNA. It’s at the heart of everything we do.
