Senior Data Engineer
Share
Job Location:
Reston, VA - USA
Monthly Salary:
$ 89203 - 151646
Posted on:
30+ days ago
Vacancies:
1 Vacancy
Job Summary
The company:
ICF is a mission-driven company filled with people who care deeply about improving the lives of others and making the world a better place. Our core values include Embracing Difference; we seek candidates who are passionate about building a culture that encourages embraces and hires dimensions of difference.
The Team:
Our Health Engineering Systems (HES) team works side by side with customers to articulate a vision for success and then make it happen. We know success doesnt happen by accident. It takes the right team of people working together on the right solutions for the customer. We are looking for a seasoned Senior Data Engineer who will be a key driver to make this happen.
Responsibilities:
- Design develop and maintain scalable data pipelines using Spark Hive and Airflow
- Develop and deploy data processing workflows on the Databricks platform
- Develop API services to facilitate data access and integration
- Create interactive data visualizations and reports using AWS QuickSight
- Builds required infrastructure for optimal extraction transformation and loading of data from various data sources using AWS and SQL technologies
- Monitor and optimize the performance of data infrastructure and processes
- Develop data quality and validation jobs
- Assembles large complex sets of data that meet non-functional and functional business requirements
- Write unit and integration tests for all data processing code
- Work with DevOps engineers on CI CD and IaC
- Read specs and translate them into code and design documents
- Perform code reviews and develop processes for improving code quality
- Improve data availability and timeliness by implementing more frequent refreshes tiered data storage and optimizations of existing datasets
- Maintain security and privacy for data at rest and while in transit
- Other duties as assigned
Minimum Qualifications:
- Bachelors degree in computer science engineering or related field
- 7 years of hands-on software or data development experience
- 4 years of data pipeline experience using Python Java and cloud technologies
- 2 years working in Spark environment
- 1 year working with Hive for big data processes
- Candidate must be able to obtain and maintain a Public Trust clearance
- Candidate must reside in the US be authorized to work in the US and work must be performed in the US
- Must have lived in the US 3 full years out of the last 5 years
- Travel of up to once a quarter US domestically is required
Preferred Qualifications:
U.S. Citizenship or Green Card is highly prioritized due to federal contract requirements
- Experience working with Copado (strongly preferred)
- Experience building job workflows with the Databricks platform
- Strong understanding of AWS products including S3 Redshift RDS EMR AWS Glue AWS Glue DataBrew Jupyter Notebooks Athena QuickSight EMR and Amazon SNS
- Familiar with work to build processes that support data transformation workload management data structures dependency and metadata
- Experienced in data governance process to ingest (batch stream) curate and share data with upstream and downstream data users.
- Experienced in data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up.
- Demonstrated understanding using software and tools including relational NoSQL and SQL databases including Cassandra and Postgres; workflow management and pipeline tools such as Airflow Luigi and Azkaban; stream-processing systems like Spark-Streaming and Storm; and object function/object-oriented scripting languages including Scala C Java and Python.
- Familiar with DevOps methodologies including CI/CD pipelines (Github Actions) and IaC (Terraform)
- Ability to obtain and maintain a Public Trust residing in the United States
- Experience with Agile methodology using test-driven development.
Job Location: This position requires that the job be performed in the United States. If you accept this position you should note that ICF does monitor employee work locations and blocks access from foreign locations/foreign IP addresses and also prohibits personal VPN connections.
#DMX-HES
#Li-cc1
#Indeed
Working at ICF
ICF is a global advisory and technology services provider but were not your typical consultants. We combine unmatched expertise with cutting-edge technology to help clients solve their most complex challenges navigate change and shape the future.We can only solve the worlds toughest challenges by building a workplace that allows everyone to thrive. We are an equal opportunity employer.Together our employees are empowered to share theirexpertiseand collaborate with others to achieve personal and professional goals. For more information please read ourEEOpolicy.
We will consider for employment qualified applicants with arrest and conviction records.
Reasonable Accommodations are available including but not limited to for disabled veterans individuals with disabilities and individuals withsincerely heldreligious beliefs in all phases of the application and employment process. To requestan accommodationplease emailand we will be happy toassist. All information you provide will be kept confidential and will be used only to the extentto provide needed reasonable accommodations.
Read more aboutworkplacediscriminationrightsor our benefit offerings which are included in theTransparency in (Benefits) CoverageAct.
At ICF we are committed to ensuring a fair interview process for all candidates based on their own skills and knowledge. As part of this commitment the use of artificial intelligence (AI) tools to generate orassistwith responses during interviews (whether in-person or virtual) is notpermitted. This policy is in place tomaintainthe integrity and authenticity of the interview process.
However we understand that some candidates may require accommodationthat involves the use of AI. Ifsuch anaccommodation is needed candidates are instructed to contact us in advance at. Weare dedicated to providingthe necessary support to ensure that all candidates have an equal opportunity to succeed.
Pay Range - There are multiple factors that are considered in determining final pay for a position including but not limited to relevant work experience skills certifications and competencies that align to the specified role geographic location education and certifications as well as contract provisions regarding labor categories that are specific to the position.
The pay range for this position based on full-time employment is:
$89203.00 - $151646.00Nationwide Remote Office (US99)Required Experience:
Senior IC
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
About ICF: The Integral Coach Factory is one of the earliest production units of independent India. It was inaugurated by the first Prime Minister of India Pt. Jawaharlal Nehru on 2nd October, 1955. Later the Furnishing Division was inaugurated on 2nd October, 1962 and the production ... View more
