Sr Data Engineer with extensive years of AWSETLData PipelinePyspark exp
Share
Job Location:
Reston, VA - USA
Monthly Salary:
Not Disclosed
Posted on:
9 hours ago
Vacancies:
1 Vacancy
Job Summary
Role : Sr Data Engineer with extensive years of AWS/ETL/Data Pipeline/Pyspark exp
Location : Remote
Term : C2C/W2/Full Time
Responsibilities:
Job Information
The Data/AI Engineer will be responsible for solution engineering of enterprise scale data management best practices. This includes patterns such as modern data integration frameworks and building of scalable distributed systems using emerging cloud-based data design patterns. This role will be responsible for developing data integration tasks in the data and analytics space. This position will report to the Director of Data Management under Data & AI organization.
The Data/AI Engineer will be responsible for solution engineering of enterprise scale data management best practices. This includes patterns such as modern data integration frameworks and building of scalable distributed systems using emerging cloud-based data design patterns. This role will be responsible for developing data integration tasks in the data and analytics space. This position will report to the Director of Data Management under Data & AI organization.
Key Job Functions
Demonstrate ability in implementing data warehouse solutions using modern data platforms such as Snowflake Databricks or Redshift.
Build data integration solutions between transaction systems and analytics platforms.
Expand data integration solutions to ingest data from internal and external sources and to further transform as per the business consumption needs.
Develop tasks for a multitude of data patterns e.g. real-time data integration advanced analytics machine learning BI and reporting.
Fundamental understanding of building of data products by data enrichment and ML.
Act as a team player and share knowledge with the existing team members.
Demonstrate ability in implementing data warehouse solutions using modern data platforms such as Snowflake Databricks or Redshift.
Build data integration solutions between transaction systems and analytics platforms.
Expand data integration solutions to ingest data from internal and external sources and to further transform as per the business consumption needs.
Develop tasks for a multitude of data patterns e.g. real-time data integration advanced analytics machine learning BI and reporting.
Fundamental understanding of building of data products by data enrichment and ML.
Act as a team player and share knowledge with the existing team members.
Qualifications:
Education
Bachelors degree in computer science or a related field.
Minimum Experience
Minimum 5 years of experience in building data driven solutions.
At least 3 years of experience working with AWS services.
Applicants must be authorized to work in the US without requiring employer sponsorship currently or in the future. U.S. FinTech does not offer H-1B sponsorship for this position.
Specialized Knowledge & Skills
Expertise in real-time data solutions good-to-have knowledge of streams processing Message Oriented Platforms and ETL/ELT Tools.
Strong scripting experience using Python and SQL.
Working knowledge of foundational AWS compute storage networking and IAM.
Understanding of Gen AI models prompt engineering RAG fine tuning and pre-tuning will be a plus.
Solid scripting experience in AWS using Lambda functions.
Knowledge of CloudFormation template preferred.
Hands-on experience with popular cloud-based data warehouse platforms such as Redshift and Snowflake.
Experience in building data pipelines with related understanding of data ingestion transformation of structured semi-structured and unstructured data across cloud services.
Knowledge and understanding of data standards and principles to drive best practices around data management activities and solutions.
Experience with one or more data integration tools such as Attunity (Qlik) AWS Glue ETL Talend Kafka etc.
Strong understanding of data security authorization authentication encryption and network security.
Hands on experience in using and extending machine learning framework and libraries e.g scikit-learn PyTorch TensorFlow XGBoost etc. preferred.
Experience with AWS SageMaker family of services or similar tools to develop machine learning models preferred.
Strong written and verbal communication skills to facilitate meetings and workshops to collect data functional and technology requirements document processes data flows gap analysis and associated data to support data management/governance related efforts.
Acts with integrity and proactively seeks ways to ensure compliance with regulations policies and procedures.
Demonstrated ability to be self-directed with excellent organization analytical and interpersonal skills and consistently meet or exceed deadline deliverables.
Strong understanding of the importance and benefits of good data quality and the ability to champion results across functions.
Education
Bachelors degree in computer science or a related field.
Minimum Experience
Minimum 5 years of experience in building data driven solutions.
At least 3 years of experience working with AWS services.
Applicants must be authorized to work in the US without requiring employer sponsorship currently or in the future. U.S. FinTech does not offer H-1B sponsorship for this position.
Specialized Knowledge & Skills
Expertise in real-time data solutions good-to-have knowledge of streams processing Message Oriented Platforms and ETL/ELT Tools.
Strong scripting experience using Python and SQL.
Working knowledge of foundational AWS compute storage networking and IAM.
Understanding of Gen AI models prompt engineering RAG fine tuning and pre-tuning will be a plus.
Solid scripting experience in AWS using Lambda functions.
Knowledge of CloudFormation template preferred.
Hands-on experience with popular cloud-based data warehouse platforms such as Redshift and Snowflake.
Experience in building data pipelines with related understanding of data ingestion transformation of structured semi-structured and unstructured data across cloud services.
Knowledge and understanding of data standards and principles to drive best practices around data management activities and solutions.
Experience with one or more data integration tools such as Attunity (Qlik) AWS Glue ETL Talend Kafka etc.
Strong understanding of data security authorization authentication encryption and network security.
Hands on experience in using and extending machine learning framework and libraries e.g scikit-learn PyTorch TensorFlow XGBoost etc. preferred.
Experience with AWS SageMaker family of services or similar tools to develop machine learning models preferred.
Strong written and verbal communication skills to facilitate meetings and workshops to collect data functional and technology requirements document processes data flows gap analysis and associated data to support data management/governance related efforts.
Acts with integrity and proactively seeks ways to ensure compliance with regulations policies and procedures.
Demonstrated ability to be self-directed with excellent organization analytical and interpersonal skills and consistently meet or exceed deadline deliverables.
Strong understanding of the importance and benefits of good data quality and the ability to champion results across functions.
Cloud BC Labs Inc is a digital transformation organization aimed at creating seamless solutions for clients to effectively manage their business operations. The company specializes in Business and Management Consulting AI/ML Data Analytics & Visualization Cloud Data Warehouse Migration Snowflake Implementation Informatica Implementation & Upgrade Staffing Services and Data Management Solutions
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
