Data Engineer Sr. Consultant
Share
Job Location:
Monthly Salary:
Posted on:
2 hours ago
Vacancies:
1 Vacancy
Job Summary
Lead architecture and delivery of largescale data platforms (ingestion transformation storage metadata lineage quality governed access)
Design highvolume batch and streaming pipelines (Spark/Flink/Kafka) with clear SLAs/SLOs observability and cost/performance targets for global transaction processing.
Build and harden backend services and APIs (REST/gRPC authN/Z rate limiting quotas resilience patterns) that power data products and model operations on Kubernetes.
Partner with Product Team to translate requirements into secure robust solutions be the teams technical representative across forums and stakeholders.
Drive proofs of concept to derisk tech choices quickly prototype then lead or hand off for productionization with best practices.
Champion engineering standards- CI/CD TDD data contracts catalog/governance PII handling RBAC/ABAC audit retention.
Establish domainoriented data models (warehouse/lakehouse/data mesh) and semantic layers enabling analytics
Own reliability for pipelines and services: metrics incident response capacity planning continuous improvement
Mentor senior engineers and onboard new joiners foster mutual accountability and effective crosstimezone collaboration.
GenAI enablement- build/enhance RAG pipelines architect/optimize AI agents for planning/reasoning/acting integrate MCP tools/clients leverage LangChain/LangGraph contribute to prompt design and finetuning.
This is a hybrid position. Expectation of days in office will be confirmed by your hiring manager.
Qualifications :
12 years in data/software engineering with significant ownership of largescale productiongrade systems Bachelors or advanced degree.
Strong backend experience: 35 years building and operating production services (Java/Scala/Go/Python Spring Boot or equivalent REST/gRPC).
Big Data engineering: Hadoop/Spark Python SQL expert in ETL/ELT and maintaining scalable Spark pipelines.
Streaming/messaging: Kafka (Connect schema registry delivery semantics) familiarity with Hazelcast or similar inmemory tech.
Containers/cloud/platform: Docker Kubernetes AWS (or GCP/Azure) InfraasCode (Terraform).
Orchestration and scheduling: Airflow ControlM (or equivalent) CI/CD with Git Jenkins/GitHub Actions Maven/Gradle.
Databases and modeling: relational (PostgreSQL/DB2) NoSQL (MongoDB/Cassandra) strong SQL and data modeling (star/snowflake 3NF domaindriven).
Distributed systems and performance: familiarity with parallel/distributed architectures tuning and evaluation for large systems.
Observability: Prometheus/Grafana/OpenTelemetry metrics and operational excellence in highthroughput environments.
Security/compliance: component security analysis IAM/leastprivilege collaboration with security teams on risk and controls.
Practices: Agile/TDD code quality initiatives ability to work independently and across teams excellent communication and stakeholder partnership.
Leadership: proven technical leadership crossorg influence mentoring passion and ability to represent the team in technical forums.
GenAI expertise: experience with RAG pipelines AI agents MCP tools/clients LangChain/LangGraph prompt crafting and contributions to model finetuning for task performance and alignment.
Additional Information :
Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race color religion sex national origin sexual orientation gender identity disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.
Remote Work :
No
Employment Type :
Full-time
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
Visa (NYSE: V) is a world leader in digital payments, facilitating transactions between consumers, merchants, financial institutions and government entities across more than 200 countries and territories. Our purpose is to uplift everyone, everywhere by being the best way to pay and b ... View more
