Senior Big Data & DevOps Engineer

Pittsburgh, PA - USA

Monthly Salary: Not Disclosed

Posted on: 30+ days ago

Vacancies: 1 Vacancy

Job Summary

Role: Senior Big Data & DevOps Engineer.

Location: Pittsburgh PA (Remote).

Duration: Long Term Contract.

Position Summary:

Seeking a Senior Big Data & DevOps Engineer with 8 years of expertise in HDFS Hive Impala PySpark Python and DevOps automation (uDeploy Jenkins). Responsible for end-to-end management of enterprise data platforms including HDFS table governance scalable ETL pipeline development CI/CD automation platform upgrades incident/change management and production support.

Key Responsibilities:

Design manage and optimize HDFS directories tables and data partitioning strategies across large distributed clusters.
Enforce data retention lifecycle policies and security compliance in high-volume Hadoop environments.
Administer Hive and Impala platforms focusing on performance tuning capacity planning and troubleshooting.
Develop scalable ETL pipelines and reusable data frameworks using PySpark Hive and Python.
Implement and support CI/CD pipelines via Jenkins and uDeploy ensuring automated reliable multi-environment deployments.
Manage platform upgrades patching and code promotion across DEV QA UAT and PROD stages.
Perform advanced Linux administration including scripting system tuning disk management and disaster recovery.
Drive incident and change management using ServiceNow including RCA investigation issue resolution and preventive actions.
Mentor junior engineers foster technical leadership and document operational knowledge for internal sharing.

Required Qualifications:

Bachelors degree in Computer Science or related field.
8 years in Big Data engineering and DevOps.
Advanced proficiency in HDFS Hive Impala PySpark Python and Linux.
Proven CI/CD experience with Jenkins and uDeploy.
Strong ETL development orchestration and optimization skills.
Experience with ServiceNow for incident/change/problem management.

Nice to Have:

Experience with cloud-based data platforms (AWS EMR).
Knowledge of Docker Kubernetes and infrastructure automation (Ansible Terraform).

Role: Senior Big Data & DevOps Engineer. Location: Pittsburgh PA (Remote). Duration: Long Term Contract. Position Summary: Seeking a Senior Big Data & DevOps Engineer with 8 years of expertise in HDFS Hive Impala PySpark Python and DevOps automation (uDeploy Jenkins). Responsible for end-to-end...