Senior Data Engineer
Share
Job Location:
Dulles Town Center, VA - USA
Monthly Salary:
Not Disclosed
Posted on:
23 hours ago
Vacancies:
1 Vacancy
Job Summary
BCMC is supporting a U.S. Government customer on a large mission critical development and sustainment program to design integrate build deliver and operate a network operations environment including introducing new cyber capabilities to address emerging threats. BCMC is seeking a Senior Data Engineer working to support the migration of customer applications services and platforms to a Medallion Model.
Responsibilities:
Data Pipeline Developer: design develop and implement end-to-end data ingestion and processing pipelines using various Cloud and Platform Technologies.
Transform incoming bronze data (RAW) into silver (clean) and finally projections into gold tier. (Medallion model)
Schedule orchestrate and monitor automated/semi-automated data pipelines.
Perform mapping and integration of data assets in support of the execution of multiple mission customers.
Develop and maintain the data engineering team process improvement plan.
Develop code using various programming and scripting languages to automate data ingestion pipeline orchestration and improve data management processes.
Ingest transparency leveraging technologies such as AWS CloudWatch to identify places for measuring and gathering performance information on automated data pipelines
Ensure Data Engineering Team Standard Operating Procedures are appropriately captured and communicated across the team.
Ensure technical correctness timeliness and quality of delivery for the team.
Demonstrate excellent oral and written communication skills to all levels of management and the customer.
Required Skills:
Must be a US Citizen
Active Secret (S) clearance. Must be able to obtain a TS/SCI clearance
Must be able to obtain DHS Suitability
8 years of software engineering experience.
Minimum of 4 years of experience performing data engineering work in a cloud environment.
Experience with relational noSQL and/or file-based storage (e.g. Databricks Elastic Postgres etc.)
Experience with DevOps tools such as Jenkins Docker Ansible Puppet Chef Splunk etc.
Working knowledge of Databricks Cloud Relational Database Services and Elasticsearch
Desired Skills:
Experience with Databricks workflows Databricks Unity Catalog Databricks Autoloader and Databricks Delta Live Tables and Apache Spark.
Basic working knowledge with Apache Niagara Files (NiFi)
Experience with collaboration tools including MS Teams MS Outlook MS SharePoint and Confluence
Amazon Web Services (AWS) Professional certification or equivalent.
Desired Certifications:
DoD 8570.1-M Compliance at IAT Level I (e.g. Certified Information Systems Security Professional (CISSP)) certification highly desired
Required Education:
Bachelors degree in Software Engineering Computer Science or a related discipline is required. Ten (10) years of experience (for a total of eighteen (18) or more years) may be substituted for a degree.
Our Company Overview
Business Computers Management Consulting Group LLC (BCMC) is a small business specializing in Information Technology (IT) Cybersecurity Information Assurance (IA) SOA Big Data Management Program Management and more for Federal State and Local agencies. We possess highly skilled engineers providing innovative solutions backed by strong past performances. We are ISO 9001:2015 ISO 27001::2018 and CMMI L3 certified and registered promising highest quality and services to all of our clients.
Benefits
Extremely competitive salary
95% employer paid for employee medical dental & vision coverages
100% employer paid for employee life STD & LTD disability coverages
401k with company match and profit sharing
Flexible Spending Account (FSA) for dependent & health care
11standard holidays & 3 weeks of annual leave
Software Engineer - Principal I - ESW05
DOM-TO9-3482
Responsibilities:
Data Pipeline Developer: design develop and implement end-to-end data ingestion and processing pipelines using various Cloud and Platform Technologies.
Transform incoming bronze data (RAW) into silver (clean) and finally projections into gold tier. (Medallion model)
Schedule orchestrate and monitor automated/semi-automated data pipelines.
Perform mapping and integration of data assets in support of the execution of multiple mission customers.
Develop and maintain the data engineering team process improvement plan.
Develop code using various programming and scripting languages to automate data ingestion pipeline orchestration and improve data management processes.
Ingest transparency leveraging technologies such as AWS CloudWatch to identify places for measuring and gathering performance information on automated data pipelines
Ensure Data Engineering Team Standard Operating Procedures are appropriately captured and communicated across the team.
Ensure technical correctness timeliness and quality of delivery for the team.
Demonstrate excellent oral and written communication skills to all levels of management and the customer.
Required Skills:
Must be a US Citizen
Active Secret (S) clearance. Must be able to obtain a TS/SCI clearance
Must be able to obtain DHS Suitability
8 years of software engineering experience.
Minimum of 4 years of experience performing data engineering work in a cloud environment.
Experience with relational noSQL and/or file-based storage (e.g. Databricks Elastic Postgres etc.)
Experience with DevOps tools such as Jenkins Docker Ansible Puppet Chef Splunk etc.
Working knowledge of Databricks Cloud Relational Database Services and Elasticsearch
Desired Skills:
Experience with Databricks workflows Databricks Unity Catalog Databricks Autoloader and Databricks Delta Live Tables and Apache Spark.
Basic working knowledge with Apache Niagara Files (NiFi)
Experience with collaboration tools including MS Teams MS Outlook MS SharePoint and Confluence
Amazon Web Services (AWS) Professional certification or equivalent.
Desired Certifications:
DoD 8570.1-M Compliance at IAT Level I (e.g. Certified Information Systems Security Professional (CISSP)) certification highly desired
Required Education:
Bachelors degree in Software Engineering Computer Science or a related discipline is required. Ten (10) years of experience (for a total of eighteen (18) or more years) may be substituted for a degree.
Our Company Overview
Business Computers Management Consulting Group LLC (BCMC) is a small business specializing in Information Technology (IT) Cybersecurity Information Assurance (IA) SOA Big Data Management Program Management and more for Federal State and Local agencies. We possess highly skilled engineers providing innovative solutions backed by strong past performances. We are ISO 9001:2015 ISO 27001::2018 and CMMI L3 certified and registered promising highest quality and services to all of our clients.
Benefits
Extremely competitive salary
95% employer paid for employee medical dental & vision coverages
100% employer paid for employee life STD & LTD disability coverages
401k with company match and profit sharing
Flexible Spending Account (FSA) for dependent & health care
11standard holidays & 3 weeks of annual leave
Software Engineer - Principal I - ESW05
DOM-TO9-3482
Required Experience:
Senior IC
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
Business Computers Management Consulting Group, LLC (BCMC) was founded with a simple purpose: to deliver excellence to our clients.
