Data Engineer

Role Responsibilities:

• Design code test debug and document technology solutions for complex projects and programs.
• Review and analyze large-scale technology solutions to align with tactical and strategic business objectives as well as enterprise technological environments.
• Evaluate and develop companywide best practices for engineering and technology solutions ensuring alignment with industry standards and new technologies.
• Collaborate with key technical experts and senior technology teams to resolve complex technical challenges and meet project goals.
• Contribute to the development of Data Mesh capabilities to improve data accessibility across disparate platforms such as Hadoop Teradata and SQL Servers.
• Support Data Science functions by preparing pre-work required for data exploration and analysis.
• Support System Testing User Acceptance Testing (UAT) and end-to-end testing activities; develop test automation for batch processes and APIs within a CI/CD environment.
• Work with Agile methodologies writing user stories and participating in Agile ceremonies.

Required Qualifications:

• Extensive experience designing and delivering data solutions in Google Cloud Platform (GCP) leveraging services such as DataProc DataFlow BigQuery and other GCP tools.
• Expertise in Big Data platforms (e.g. MapR Cloudera) using technologies like HDFS Spark PySpark and Hive.
• Proficiency in event-driven solutions using Java REST APIs Apache Flink and Kafka technologies.
• Experience with relational and NoSQL databases including MongoDB SQL Server Atlas and Azure SQL.
• Strong familiarity with wire formats such as XML JSON Avro and CSV along with serialization/deserialization techniques.
• Knowledge of security protocols (e.g. Kerberos JWT) and their integration into data platforms and solutions.
• Solid understanding of Agile methodologies including writing user stories and participating in ceremonies such as sprint planning and retrospectives.