Data Engineer

Job Summary

Synechron is seeking a highly skilled and experienced Senior Data Engineer to join our innovative analytics team in Bangalore. The primary purpose of this role is to design develop and maintain scalable data pipelines and architectures that empower data-driven decision making and advanced analytics initiatives. As a critical contributor within our data ecosystem you will enable the organization to harness large complex datasets efficiently supporting strategic business objectives and ensuring high standards of data quality security and performance. Your expertise will directly contribute to building robust efficient and secure data solutions that drive business value across multiple domains.

Software Requirements

Required Software & Tools:

• Databricks Platform (Hands-on experience with Databricks notebooks clusters and workflows)
• PySpark (Proficient in developing and optimizing Spark jobs)
• SQL (Advance proficiency in writing complex SQL queries and optimizing queries)
• Data Orchestration Tools such as Apache Airflow or similar (Experience in scheduling and managing data workflows)
• Cloud Data Platforms (Experience with cloud environments such as AWS Azure or Google Cloud)
• Data Warehousing Solutions (Snowflake highly preferred)

Preferred Software & Tools:

• Kafka or other streaming frameworks (e.g. Confluent MQTT)
• CI/CD tools for data pipelines (e.g. Jenkins GitLab CI)
• DevOps practices for data workflows
• Programming Languages: Python (Expert level) and familiarity with other languages like Java or Scala is advantageous

Overall Responsibilities

• Architect develop and maintain scalable resilient data pipelines and architectures supporting business analytics reporting and data science use cases.
• Collaborate closely with data scientists analysts and cross-functional teams to gather requirements and deliver optimized data solutions aligned with organizational goals.
• Ensure data quality consistency and security across all data workflows adhering to best practices and compliance standards.
• Optimize data processes for enhanced performance reliability and cost efficiency.
• Integrate data from multiple sources including cloud data services and streaming platforms ensuring seamless data flow and transformation.
• Lead efforts in performance tuning and troubleshooting data pipelines to resolve bottlenecks and improve throughput.
• Stay up-to-date with emerging data engineering technologies and contribute to continuous improvement initiatives within the team.

Technical Skills (By Category)

Programming Languages:

• Essential: Python SQL
• Preferred: Scala Java

Databases/Data Management:

• Essential: Data modeling ETL/ELT processes data warehousing (Snowflake experience highly preferred)
• Preferred: NoSQL databases Hadoop ecosystem

Cloud Technologies:

• Essential: Experience with cloud data services (AWS Azure GCP) and deployment of data pipelines in cloud environments
• Preferred: Cloud native data tools and architecture design

Frameworks and Libraries:

• Essential: PySpark Spark SQL Kafka Airflow
• Preferred: Streaming frameworks TensorFlow (for data prep)

Development Tools and Methodologies:

• Essential: Version control (Git) CI/CD pipelines Agile methodologies
• Preferred: DevOps practices in data engineering containerization (Docker Kubernetes)

Security Protocols:

• Familiarity with data security encryption standards and compliance best practices

Experience Requirements

• Minimum of 8 years of professional experience in Data Engineering or related roles
• Proven track record of designing and deploying large-scale data pipelines using Databricks PySpark and SQL
• Practical experience in data modeling data warehousing and ETL/ELT workflows
• Experience working with cloud data platforms and streaming data frameworks such as Kafka or equivalent
• Demonstrated ability to work with cross-functional teams translating business needs into technical solutions
• Experience with data orchestration and automation tools is highly valued
• Prior experience in implementing CI/CD pipelines or DevOps practices for data workflows (preferred)

Day-to-Day Activities

• Design develop and troubleshoot data pipelines for ingestion transformation and storage of large datasets
• Collaborate with data scientists and analysts to understand data requirements and optimize existing pipelines
• Automate data workflows and improve pipeline efficiency through performance tuning and best practices
• Conduct data quality audits and ensure data security protocols are followed
• Manage and monitor data workflows troubleshoot failures and implement fixes proactively
• Contribute to documentation code reviews and knowledge sharing within the team
• Stay informed of evolving data engineering tools techniques and industry best practices incorporating them into daily work processes

Qualifications

• Bachelors or Masters degree in Computer Science Information Technology or related field
• Relevant certifications such as Databricks Certified Data Engineer AWS Certified Data Analytics or equivalent (preferred)
• Continuous learning through courses workshops or industry conferences on data engineering and cloud technologies

Professional Competencies

• Strong analytical and problem-solving skills with a focus on scalable solutions
• Excellent communication skills to effectively collaborate with technical and non-technical stakeholders
• Ability to prioritize tasks manage time effectively and deliver within tight deadlines
• Demonstrated leadership in guiding team members and driving project success
• Adaptability to evolving technological landscapes and innovative thinking
• Commitment to data privacy security and ethical handling of information

Day-to-Day Activities

• Design develop and troubleshoot data pipelines for ingestion transformation and storage of large datasets
• Collaborate with data scientists and analysts to understand data requirements and optimize existing pipelines
• Automate data workflows and improve pipeline efficiency through performance tuning and best practices
• Conduct data quality audits and ensure data security protocols are followed
• Manage and monitor data workflows troubleshoot failures and implement fixes proactively
• Contribute to documentation code reviews and knowledge sharing within the team
• Stay informed of evolving data engineering tools techniques and industry best practices incorporating them into daily work processes

Qualifications

• Bachelors or Masters degree in Computer Science Information Technology or related field

• Relevant certifications such as Databricks Certified Data Engineer AWS Certified Data Analytics or equivalent (preferred)
• Continuous learning through courses workshops or industry conferences on data engineering and cloud technologies

Professional Competencies

• Strong analytical and problem-solving skills with a focus on scalable solutions
• Excellent communication skills to effectively collaborate with technical and non-technical stakeholders
• Ability to prioritize tasks manage time effectively and deliver within tight deadlines
• Demonstrated leadership in guiding team members and driving project success
• Adaptability to evolving technological landscapes and innovative thinking
• Commitment to data privacy security and ethical handling of information

SYNECHRONS DIVERSITY & INCLUSION STATEMENT

Diversity & Inclusion are fundamental to our culture and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity Equity and Inclusion (DEI) initiative Same Difference is committed to fostering an inclusive culture promoting equality diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger successful businesses as a global company. We encourage applicants from across diverse backgrounds race ethnicities religion age marital status gender sexual orientations or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements mentoring internal mobility learning and development programs and more.

All employment decisions at Synechron are based on business needs job requirements and individual qualifications without regard to the applicants gender gender identity sexual orientation race ethnicity disabled or veteran status or any other characteristic protected by law.

Candidate Application Notice