Enter a job title or keyword
Senior Python Engineer – Data Pipelines
Job Location:
Job Location:
Jersey, NJ - USA
Monthly Salary:
Not provided by the employer
Monthly Salary:
Posted:
4 November 2026 (Yesterday)
Posted:
Application Deadline:
1 February 2027
Application Deadline:
Vacancies:
1 Vacancy
Vacancies:
Job Summary
This is a hybrid role based in Hyderabad India.
Role Summary
Seeking a hands-on senior Python engineer to own data pipelines end to end (ingestion transformation orchestration quality and delivery) at enterprise scale. Pipelines feed analytics search and GenAI/Agentic AI platforms.
Responsibilities
- Design build and own production data pipelines in Python (batch and near-real-time).
- Operate orchestrated workflows (AWS Step Functions Airflow Dagster or Prefect) with retries idempotency and backfills.
- Model data at scale: partitioning Parquet/Avro/Iceberg schema evolution CDC.
- Establish data quality and observability (Great Expectations/Pandera SLAs lineage alerting).
- Build document-processing and RAG retrieval pipelines (chunking embeddings vector and hybrid search).
- Build serving APIs with FastAPI Flask or Django REST.
- Set Python engineering standards; mentor junior engineers; liaise with business stakeholders.
Qualifications
- 812 years of software engineering 56 years building production data pipelines in Python.
- Expert modern Python (3.10) typing asyncio/concurrency pandas/Polars PyArrow PySpark/Dask/Ray.
- Strong SQL and data modeling; Snowflake Redshift Iceberg/Delta dbt.
- Strong AWS: Lambda Step Functions S3 Glue EMR IAM CloudWatch; IaC (CDK/Terraform).
- pytest packaging (uv/Poetry) CI/CD.
- Strongly preferred: RAG AWS Bedrock LangChain/LlamaIndex agentic frameworks MCP servers GenAI evaluation.
- Nice to have: Kafka/Kinesis/Flink streaming C#/.NET regulated-industry experience.
