Data Engineering Consultant
Data Engineering Consultant
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Role Overview
A Data Engineering Consultant designs implements and optimizes scalable data pipelines and architectures. This role bridges raw data and actionable insights ensuring robustness performance and data governance. Collaboration with analysts and scientists is central to delivering high-quality solutions aligned with business objectives.
Key Responsibilities
- Data Pipeline Development
- Architect implement and maintain real-time and batch data pipelines to handle large datasets efficiently.
- Employ frameworks such as Apache Spark Databricks Snowflake or Airflow to automate ingestion transformation and delivery.
- Data Integration & Transformation
- Work with Data Analysts to understand source-to-target mappings and quality requirements.
- Build ETL/ELT workflows validation checks and cleaning steps for data reliability.
- Automation & Process Optimization
- Automate data reconciliation metadata management and error-handling procedures.
- Continuously refine pipeline performance scalability and cost-efficiency.
- Collaboration & Leadership
- Coordinate with Data Scientists Data Architects and Analysts to ensure alignment with business goals.
- Mentor junior engineers and enforce best practices (version control CI/CD for data pipelines).
- Participate in technical presales activities and client engagement initiatives.
- Governance & Compliance
- Apply robust security measures (RBAC encryption) and ensure regulatory compliance (GDPR).
- Document data lineage and recommend improvements for data ownership and stewardship.
Qualifications :
- Programming: Python SQL Scala Java.
- Big Data: Apache Spark Hadoop Databricks Snowflake etc.
- Cloud: AWS (Glue Redshift) Azure (Synapse Data Factory Fabric) GCP (BigQuery Dataflow).
- Data Modelling & Storage: Relational (PostgreSQL SQL Server) NoSQL (MongoDB Cassandra) Dimensional modelling.
- DevOps & Automation: Docker Kubernetes Terraform CI/CD pipelines for data flows.
- Architectural Competencies
- Data Modelling: Designing dimensional relational and hierarchical data models.
- Scalability & Performance: Building fault-tolerant highly available data architectures.
- Security & Compliance: Enforcing role-based access control (RBAC) encryption and auditing.
Additional Information :
Discover some of the global benefits that empower our people to become the best version of themselves:
- Finance: Competitive salary package share plan company performance bonuses value-based recognition awards referral bonus;
- Career Development: Career coaching global career opportunities non-linear career paths internal development programmes for management and technical leadership;
- Learning Opportunities: Complex projects rotations internal tech communities training certifications coaching online learning platforms subscriptions pass-it-on sessions workshops conferences;
- Work-Life Balance: Hybrid work and flexible working hours employee assistance programme;
- Health: Global internal wellbeing programme access to wellbeing apps;
- Community: Global internal tech communities hobby clubs and interest groups inclusion and diversity programmes events and celebrations.
At Endava were committed to creating an open inclusive and respectful environment where everyone feels safe valued and empowered to be their best. We welcome applications from people of all backgrounds experiences and perspectivesbecause we know that inclusive teams help us deliver smarter more innovative solutions for our customers. Hiring decisions are based on merit skills qualifications and potential. If you need adjustments or support during the recruitment process please let us know.
Remote Work :
No
Employment Type :
Full-time
Employment Type
Full-time
