Senior Data Engineer (AI)

Estaffing Inc

Job Location:

Pittsburgh, PA - USA

Monthly Salary: Not Disclosed

Posted on: 9 days ago

Vacancies: 1 Vacancy

Job Summary

Job Title: Senior Data Engineer (AI)-HYBRID

Job Type: Contractextension

Location: Pittsburgh PA

Job Duration: Contract through end of 2026 (extension eligible)

Note: Contract through end of 2026 (extension eligible)

!!NO C2C only W2!!

Please read the job description carefully Before APPLY!

Job Description:

We are seeking a Senior Data Engineer with strong expertise in modern data engineering and working knowledge of MLOps practices to support the Knowledge Navigator platform. This individual will play a critical role in building and maintaining scalable data pipelines that feed downstream analytics and AI/ML use cases.

The ideal candidate is a hands-on engineer who thrives in Azure-based environments is experienced with large-scale data processing using Databricks and can operate effectively within established enterprise architectures. This role requires a strong focus on data reliability pipeline performance and adherence to enterprise data governance and security standards while collaborating across engineering analytics and AI teams.

Responsibilities

Design build and maintain scalable data ingestion and processing pipelines into Azure Data Lake Storage (ADLS) and Databricks
Implement batch and/or streaming data pipelines to support analytics and AI/ML workloads
Develop and optimize data transformations using Spark (PySpark) and SQL within Databricks
Integrate data from enterprise source systems (e.g. APIs relational databases SaaS platforms) into centralized data platforms
Ensure high data quality integrity and availability for downstream consumers
Monitor troubleshoot and optimize pipeline performance reliability and cost efficiency
Support pipeline automation orchestration and scheduling using tools such as Azure Data Factory or Synapse Pipelines
Collaborate with data scientists and ML engineers to support data and model pipeline workflows
Implement logging monitoring and alerting to proactively identify issues in production pipelines
Adhere to enterprise data governance security and access control policies
Work within established architectural frameworks while contributing improvements and best practices
Partner with cross-functional teams to ensure stable and scalable data operations

Requirements-

7 years of professional experience (post-graduate) in data engineering or related field
Strong hands-on experience with Azure data services including:
Azure Data Lake Storage (ADLS)
Azure Data Factory and/or Synapse Pipelines
Deep experience with Databricks including Spark / PySpark development and performance optimization
Proven experience building and maintaining enterprise-scale data pipelines
Experience integrating data from diverse enterprise systems (databases APIs cloud platforms)
Solid understanding of data architecture principles data modeling and ETL/ELT patterns
Experience with pipeline orchestration automation and monitoring
Working knowledge of MLOps concepts including supporting data workflows for model training and deployment
Strong troubleshooting and performance tuning skills in distributed environments
Familiarity with data governance security and access control frameworks
Proficiency in Python and SQL
Strong collaboration and communication skills

Experience supporting AI/ML or GenAI platforms and workflows

Familiarity with CI/CD pipelines for data engineering or ML workflows

Experience with real-time/streaming data technologies (e.g. Kafka Azure Event Hubs)

Job Title: Senior Data Engineer (AI)-HYBRID Job Type: Contractextension Location: Pittsburgh PA Job Duration: Contract through end of 2026 (extension eligible) Note: Contract through end of 2026 (extension eligible) !!NO C2C only W2!! Please read the job description carefully Befor...