Sr. Data Engineer

Pune - India

Monthly Salary: INR 2500000 - 2500000

Experience Required: 6years

Posted on: 4 hours ago

Vacancies: 1 Vacancy

Job Summary

We are seeking a highly skilled Senior Data Engineer with 78 years of experience to design develop and optimize data pipelines and solutions. The ideal candidate will have expertise in PySpark SQL and Python with solid knowledge of Azure Databricks. A strong understanding of agile methodologies will be an added advantage.

Key Responsibilities:

Design build and maintain scalable data pipelines and ETL processes.
Work with large datasets to ensure high data quality accuracy and performance.
Develop data solutions leveraging Azure Databricks PySpark and SQL.
Collaborate with cross-functional teams including Data Scientists Analysts and Product teams to support data-driven initiatives.
Troubleshoot and optimize data workflows to improve reliability and efficiency.
Ensure compliance with best practices in data governance security and performance.
Contribute to agile ceremonies and work in a collaborative fast-paced environment.

Required Skills & Qualifications:

78 years of experience in Data Engineering.
Strong programming skills in Python and SQL.
Hands-on experience with PySpark for large-scale data processing.
Good working knowledge of Azure Databricks.
Strong problem-solving and debugging skills.
Experience in designing and optimizing ETL pipelines.
Knowledge of Agile methodologies (Scrum/Kanban) is a plus.

Nice to Have:

Familiarity with other Azure services such as Data Lake Synapse or Data Factory.
Exposure to CI/CD practices and version control systems (e.g. Git).
Experience working in a cloud-first data ecosystem.

Required Skills:

Requirements We are seeking a visionary AI Architect to lead the design development and implementation of cutting-edge AI solutions. The ideal candidate will have a strong background in machine learning data engineering cloud architecture and enterprise-grade solution design. You will collaborate with stakeholders to translate business challenges into scalable AI-driven systems. Key Responsibilities: Lead the design and architecture of AI/ML solutions across the organization. Collaborate with data scientists engineers product owners and business leaders to define AI use cases and deliverables. Define architecture standards and best practices for AI solution development and deployment. Ensure AI models are scalable robust and integrated with existing systems and data platforms. Oversee end-to-end MLOps processes including versioning monitoring and continuous integration/deployment (CI/CD). Evaluate and select AI tools frameworks and technologies aligned with enterprise needs. Stay current on AI/ML trends tools and regulations (including ethical AI practices). Provide technical leadership and mentorship to development teams. Prepare architecture documentation and presentations for stakeholders and leadership. Required Qualifications: Bachelors or Masters degree in Computer Science Data Science Artificial Intelligence or related field. Proven experience in architecting AI/ML solutions in cloud environments (AWS Azure or GCP). Deep understanding of machine learning algorithms NLP computer vision and deep learning frameworks (e.g. TensorFlow PyTorch Hugging Face). Strong proficiency in Python and relevant ML libraries. Solid experience with data engineering pipelines data lakes and APIs. Familiarity with MLOps platforms (e.g. MLflow Kubeflow SageMaker Azure ML). Strong knowledge of software architecture patterns and microservices. Excellent communication stakeholder management and documentation skills. Preferred Qualifications: AI/ML certification (e.g. Google Professional ML Engineer AWS Machine Learning Specialty). Experience working in regulated industries (healthcare finance etc.). Familiarity with Generative AI LLMs (like GPT) and Responsible AI principles. Experience in DevOps or Infrastructure as Code (e.g. Terraform Docker Kubernetes).

Required Education:

Any Professional Degree