IT Data Engineer

We are seeking an experienced Senior IT Data Engineer to support a high-impact long-term project based in Washington DC. This role requires a strong background in data engineering data pipeline development and working with large-scale distributed data systems.

This is an onsite position and ideal for candidates who are local to the DC area.

The Senior IT Data Engineer will play a key role in designing building and maintaining scalable data pipelines and data platforms. This role involves working with complex data systems to ensure high-quality data processing integration and accessibility across multiple sources.

Design develop and maintain robust data pipelines and data processing systems
Build and optimize ETL workflows for large-scale structured and unstructured data
Ensure data quality integrity and performance across multiple data sources
Integrate data from various platforms and formats into centralized systems
Troubleshoot and resolve data flow and pipeline performance issues
Collaborate with cross-functional teams to support data-driven initiatives
Participate in Agile ceremonies (Scrum/Kanban) and contribute to continuous improvement
Support CI/CD pipelines and automation for data platform deployment
Maintain documentation for data processes workflows and systems

Minimum 10 years of overall IT experience
5 years of experience in data/application development (Python preferred)
Strong experience building and managing data pipelines on Cloudera Data Platform
Experience with ETL processes data transformation and performance tuning
Hands-on experience with tools like PySpark Pandas or dbt
Experience with data ingestion/integration tools such as Apache NiFi
Advanced knowledge of SQL Java and Microsoft SQL Server
Experience with distributed data technologies such as:

• Hadoop
• MapReduce
• Hive
• HBase
• Kafka
• Spark
  Experience working in UNIX/Linux environments including shell scripting
  Familiarity with CI/CD pipelines and DevOps practices
  Experience working in Agile environments (Scrum/Kanban)

Experience supporting government or regulated environments
Strong understanding of data governance security and compliance practices