Data Engineering Lead

Job Title: Data Engineering Lead

Reporting To: Head of Data Engineering

Business: Technology and Innovation

Team: Technology Data Engineering

Location: India

Lloyds List Intelligence has proven success across the globe with offices in APAC (Singapore China Hong Kong and Japan) EMEA (UK UAE) and USA. We now look to further expand our global footprint and add a location that gives us the capability to build teams of highly talented colleagues and deliver growth in our Technology Data and Product functions.

Role Profile

Reporting into the Head of Data Engineering we are seeking a Data Engineer Lead to join our team of existing Data Engineers to assist with the ongoing maintenance and innovation of our data processing and ingestion pipelines and systems.

The position presents an exciting opportunity for an experienced Data Engineer to manage and technically lead a team of Data Engineers responsible for delivering project deliverables involving back-end development maintenance of our data ingestion streaming and analytical systems. The team is tasked with ensuring our various data ingestion systems achieve a high level of reliability and work closely with our product data science and software engineering teams to support existing and new data workflows.

This is a full-time permanent role reporting to the Head of Data Engineering. This role is balanced working in our Chennai office.

Key Responsibilities

• Management of the global Data Engineering team this includes training coaching mentoring and recruitment
• Driving the team and individuals objectives according to the KPIs
• Hands-on implementation and delivery of data solutions
• Collaborate with Product teams ensuring that raw data is cleansed and transformed and useable by downstream consumers (Data Scientists ML Engineers BI analytics)
• Assist and advise on the re-development and modernisation of end-to-end ETL pipelines and introduce new technologies where appropriate in a real-time streaming environment dealing with very large data volumes
• Working with AWS cloud infrastructure (specifically SQS SNS Redshift OpenSearch Athena DynamoDB Kinesis step functions lambda) to develop innovate and maintain data flowing through various queues and data warehouses
• Working with a variety of data repository platforms (including SQL stores such as Oracle) as well as implementing data visualisation and network analysis (e.g GraphDB)
• Maintain and productionise machine learning and AI models
• Assist in the creation of next generation data ingestion platforms sourcing data using Webscrapes APIs Email and flat file (FTP) methods
• Understanding conflict resolution methods and assist subject matter experts in the debugging on data ingestion and managing overall feed uptimes across a large set of data collectors.
• Create and maintain detailed documentation and functional design specifications including data flows and data conversion
• Provide technical information to assist in the development of client facing product documentation
• Adhere to change management protocols and version control
• Present advanced technical designs with non-technical stakeholders

What we are looking for:

Essential

• At least 5 years experience as a Senior or Lead Data Engineer
• Proven and demonstratable experience of Python AWS and SQL/ Oracle
• Bachelors honours degree preferably in computer science or engineering field or demonstrable technical experience in a related engineering discipline
• At least 3 years of ETL experience using flat files to transposed Data Models
• Experience in troubleshooting and debugging Python SQL (Oracle & other SQL platforms) SNS SQS Kinesis C AWS Lambda
• Strong Data Mining and Analytical skills required to encode sophisticated business transformation logic in a pipeline and serve predictive models
• Familiarity with the design development and maintenance of best-in-class BI capabilities including data warehouse data structures and data pipelines (Redshift Athena etc)
• Familiarity of architectural design patterns for micro-services leveraging relational and big data technologies
• Familiarity with Agile development process
• Exceptional analytical thinking ability ease with quantitative analysis and excellent problem-solving skills
• Self discipline and willingness to learn
• Ability to make connections with technical and non-technical stakeholders
• Excellent verbal and written communication skills

Desirable

• Previous experience of managing a team
• Domain knowledge or experience working within the maritime or an adjacent industry
• Previous experience using C# C and JAVA

• Experience of streaming or real-time processing at high data volumes
• Familiarity of architectural design patterns for micro-services leveraging relational and big data technologies

• Demonstrable experience working with Big Data environments

• Experience of the SAS Analytics platform of particular interest

We know that sometimes the perfect candidate doesnt exist and people can be put off from applying for a job if they dont fit all the requirements. If youre excited about working for us and have most but not all of the skills or experience were looking for please go ahead and apply. You could be just what we need!

We believe strongly in the value of diversity and in creating supportive inclusive environments where our colleagues can succeed. As such we are proud to be an Equal Opportunity Employer. We do not discriminate based on race color ancestry national origin religion mental or physical disability medical condition genetic information sex (including pregnancy childbirth and related medical conditions) sexual orientation gender identity gender expression age marital status military or veteran status citizenship or other protected characteristics under federal state or local law.

Please let us know of any adjustments we need to make for you to show us your best self.

About us

We are the industry experts delivering actionable maritime insight data technology and analytics trusted by 60000 professionals across the globe to drive commercial advantage evaluate risk and support the efficient and lawful movement of seaborne trade.

Our data is collected only from the most reputable partners and accurate sources many of which are exclusive to us and our customers.

Our advanced analytics artificial intelligence and industry expertise transform our unparalleled data into powerful insight delivered through data and analytics platforms and services news and commentary and publications that help professionals in maritime operations risk and compliance stay well informed.

Leading seaborne trade through change with data transparency innovative technology and human ingenuity.

Our Values and Behaviours

For LLI these arent just the wallpaper. These are the behavioural expectations led and role modelled by our CEO Michael Dell and they are also the design principles of an change or initiative we put in place.

Build trust & Transparency

Transparency and trust needs to be at the heart with commitment to honest communication about

individual progress as well as company news regular townhall meetings clear messaging about the

companys direction and transparency in addressing internal challenges such as gender and race

inequality and recognition/reward.

Embrace collective brilliance

Collaboration and teamwork are the foundation of our collective success. Our data highlights the importance of

working together cross-company and communicating with agility to solve problems. There is a strong emphasis

on friendly authentic relationships and courageous collaboration.

Work with respect

Respect for one anothers well-being fairness in recognition of effort inclusivity in approach and

celebration of cultural differences is critical. There is emphasis placed on social awareness and

positive impact.

Take responsibility

The importance of being accountable to each other cannot be underestimated recognizing each

others work and maintaining a challenge culture to ensure credibility dependability and

trustworthiness. There is focus on stepping up to take on more if it helps the collective organization.

Simplify the complex

There is a focus in our data on delivering excellence recognizing and learning from failures and

a commitment to delivering work that puts customers first and provides market-leading

expertise.

Move fast & be bold

The data sources emphasize the need to embrace change in an agile way with a positive approach to

challenges and transformation. It includes a desire to evolve and remain industry leaders embracing

challenges and promoting a culture of curiosity and continuous learning.