Senior Data Engineer
Location: Ashburn VA
Long Term Project
Note:Must hold or be able to obtain CBP Public Trust Clearance.
10 years ETL Spark/Databricks AWS Data Engineering
Key Responsibilities:
-
Design develop and maintain scalable ETL pipelines using Databricks and Apache Spark
-
Migrate and optimize big data workflows in AWS.
-
Implement data governance quality assurance and performance enhancements
-
Work with large datasets (80 TB) performing analysis and producing regular/ad hoc reports
-
Support cloud/on prem migrations and source code control via Git/GitLab
-
Mentor junior data engineers and help establish best practices
Required Experience & Skills:
-
10 years in ETL with Spark and Databricks.
-
Proven AWS Data Engineering experience.
-
Strong SQL Python/Scala skills.
-
Experience with Unix/Linux shell scripting CRON.
-
Familiarity with data warehousing (e.g. Oracle Redshift).
-
Comfortable working with big data volumes (80 TB)
-
Excellent problem-solving and stakeholder collaboration.