We are looking for a Senior Data Engineer to design and develop data pipelines that will be the basis of our Knowledge Graph. The Knowledge Graph aims to unify insights on data processing and lineage across the entire Services division pulling in metadata on workflows execution jobs schemas and more to generate a rich view of our data pipelines exposed in an application that will be used daily by hundreds of internal data engineers. The graph will also power operational excellence capturing production issues driving analysis and guiding resolution of issues and enabling proactive avoidance of future work alongside a Dublin based team of software and other data engineers committed to bringing the knowledge graph to life focusing on designing the data layer. You will have significant individual responsibility and influence over the direction of this critical service and the opportunity to learn and interact with a department of global teams each with unique skill sets and operating in different time zones.
Bachelors degree in Computer Science Computer Engineering Mathematics or equivalent professional experience.
Industry experience in a Data Engineering/Data Architecture role
Experience working with Spark and other distributed data technologies (e.g. Hadoop Presto Druid) for building efficient & large scale data pipelines.
Experience in designing streaming data pipelines (eg Kafka Flink Spark-Streaming)
Growth mindset and ability to learn new technologies
Experience in backend API development and management
Experience with a graph database (eg Neo4j TigerGraph)
Familiar with Data Lake and Data Warehouse technologies
Good understanding of software development life cycle version control code reviews testing data quality tools and frameworks
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.