Data Engineer

Role Overview and Expected Deliverables

The engineer will design build and operate scalable secure and analyticsready data pipelines and models across GCP BigQuery and AWS delivering:
- Airfloworchestrated data workflows and automations.
- DBT Cloud transformations testing frameworks and environment governance.
- Robust CI/CD and Infrastructure as Code (IaC) deployments (GitHub Actions Bitbucket Bamboo Terraform or CloudFormation).
- Performance optimisation across data pipelines SQL BigQuery and AWS data services.
- Highquality technical documentation production support incident response and handover to internal teams.
- Close collaboration with crossfunctional stakeholders including analytics product and platform engineering.

Qualifications :

Must have skills

GCP BigQuery Cloud Storage Apache Airflow dbt Python SQL data pipelines ingestion transformation Git/Bitbucket data quality checks

Good to have skills

Dataplex Collibra integration CI/CD Grafana Splunk Power BI data preparation financial services data

Core Technical Requirements

Cloud & Platform Engineering:
- Strong proficiency across AWS and GCP.
- On AWS: IAM roles and policies Lambda S3 RDS Redshift.
- On GCP: BigQuery workload optimisation cost management.

Data Engineering Tooling:
- Handson experience building data pipelines for data warehouse environments.
- Strong SQL capability including analytical and window functions.
- Advanced dbt Cloud experience (models SCDs tests macros release management).
- Airflow workflow design including idempotency backfills and monitoring.

DevOps CI/CD & IaC:
- Experience with CI/CD (GitHub Bitbucket GitHub Actions Bamboo).
- Infrastructure as Code using Terraform or CloudFormation.
- Familiarity with data quality frameworks and automated testing of pipelines.

Programming & Modelling:
- Proficiency in Python.
- Strong data modelling expertise: facts/dimensions SCD patterns performance optimisation.

Security Data Governance & Operational Resilience:
- Solid grounding in cloud security best practices and leastprivilege IAM.
- Understanding of data access controls regulated environments and governance expectations.

Delivery Approach & Ways of Working- Experience working within Agile delivery frameworks.
- Use of JIRA and Confluence for delivery management and documentation.
- Proven track record collaborating effectively in multidisciplinary teams across engineering product and analytics.

Specialisations - BigQuery performance tuning and cost optimisation.
- DBT Cloud governance (contracts tests macros release processes).
- Airflow workflow optimisation and monitoring.
- Crosscloud pipelines and data movement (AWS GCP).
- Security IAM and dataaccess design for regulated financial services environments.

Experience & Other- 8 or more years experience in data engineering or equivalent technical roles.
- Strong problemsolving capability comfortable troubleshooting pipeline failures and performance issues.
- Strong communication and stakeholder management skills.
- Selfstarter able to lead components independently while partnering closely with internal teams.
- Experience supporting production environments including incident response and rootcause analysis.
- All intellectual property rights retained by client

Additional Information :

Open to hire for Gurgaon and Hyderabad location as well

At Endava were committed to creating an open inclusive and respectful environment where everyone feels safe valued and empowered to be their best. We welcome applications from people of all backgrounds experiences and perspectivesbecause we know that inclusive teams help us deliver smarter more innovative solutions for our customers. Hiring decisions are based on merit skills qualifications and potential. If you need adjustments or support during the recruitment process please let us know.

Remote Work :

No

Employment Type :

Full-time