Data Engineer
Share
Job Location:
St. Louis, MO - USA
Monthly Salary:
Not Disclosed
Posted on:
Yesterday
Vacancies:
1 Vacancy
Job Summary
ATTENTION MILITARY AFFILIATED JOB SEEKERS - Our organization works with partner companies to source qualified talent for their open roles. The following position is available to Veterans Transitioning Military National Guard and Reserve Members Military Spouses Wounded Warriors and their Caregivers. If you have the required skill set education requirements and experience please click the submit button and follow the next steps.
As a Data Engineer you will play a crucial role in the evolution of our data platform support the activation of AI capabilities across the team and build data products and self-service solutions for the users of our data platform.
This role requires being in the office 3 days/week on Tues - Thurs.
This position does not offer immigration sponsorship (current or future) including F-1 STEM OPT extension support.
What youll do:
Build complex batch and streaming pipelines to ingest data from upstream Equifax cloud systems.
Design and implement data engineering frameworks to scale the development and deployment of data pipelines across the D&A organization.
Leverage AI-powered coding assistants to accelerate development optimize code and generate documentation for data pipelines and infrastructure.
Develop and refine prompts for Large Language Models (LLMs) to assist in data-related tasks such as data cleansing transformation logic generation and automated data documentation.
Design build and maintain scalable data pipelines that support AI/ML applications.
Minimum Qualifications:
At least 5 years of experience in data engineering data architecture or a related field.
A strong understanding of data engineering principles and best practices including data modeling data warehousing and data integration.
At least 1 year of experience working in a GCP big data environment.
Experience building complex data pipelines and solutions using two or more of the following: BigQuery DataFlow DataProc Pub/Sub Cloud Functions.
Experience with Airflow or Cloud Composer.
Experience with Vertex AI.
Proficiency in Python development.
Professional experience with SQL.
Proven ability to effectively communicate complex technical concepts to both technical and non-technical stakeholders.
A Bachelors degree or higher in Computer Science Information Systems or a related field.
Preferred Qualifications:
Demonstrated experience with prompt engineering to interact with and guide AI models for optimal outputs.
Hands-on experience with AI-assisted coding tools
Familiarity with frameworks for building AI applications or agents (e.g. LangChain LlamaIndex).
Experience integrating AI services and models into data pipelines via APIs
DevOps/CICD experience (Jenkins and GitHub) and IaC experience using Terraform for managing GCP infrastructure.
GCP Associate Cloud Engineer or Professional Data Engineer certification.
As a Data Engineer you will play a crucial role in the evolution of our data platform support the activation of AI capabilities across the team and build data products and self-service solutions for the users of our data platform.
This role requires being in the office 3 days/week on Tues - Thurs.
This position does not offer immigration sponsorship (current or future) including F-1 STEM OPT extension support.
What youll do:
Build complex batch and streaming pipelines to ingest data from upstream Equifax cloud systems.
Design and implement data engineering frameworks to scale the development and deployment of data pipelines across the D&A organization.
Leverage AI-powered coding assistants to accelerate development optimize code and generate documentation for data pipelines and infrastructure.
Develop and refine prompts for Large Language Models (LLMs) to assist in data-related tasks such as data cleansing transformation logic generation and automated data documentation.
Design build and maintain scalable data pipelines that support AI/ML applications.
Minimum Qualifications:
At least 5 years of experience in data engineering data architecture or a related field.
A strong understanding of data engineering principles and best practices including data modeling data warehousing and data integration.
At least 1 year of experience working in a GCP big data environment.
Experience building complex data pipelines and solutions using two or more of the following: BigQuery DataFlow DataProc Pub/Sub Cloud Functions.
Experience with Airflow or Cloud Composer.
Experience with Vertex AI.
Proficiency in Python development.
Professional experience with SQL.
Proven ability to effectively communicate complex technical concepts to both technical and non-technical stakeholders.
A Bachelors degree or higher in Computer Science Information Systems or a related field.
Preferred Qualifications:
Demonstrated experience with prompt engineering to interact with and guide AI models for optimal outputs.
Hands-on experience with AI-assisted coding tools
Familiarity with frameworks for building AI applications or agents (e.g. LangChain LlamaIndex).
Experience integrating AI services and models into data pipelines via APIs
DevOps/CICD experience (Jenkins and GitHub) and IaC experience using Terraform for managing GCP infrastructure.
GCP Associate Cloud Engineer or Professional Data Engineer certification.
Required Experience:
IC
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
VetJobs & Military Spouse Jobs works with our employer partners to source, screen, and move qualified talent to the desktops of the Hiring Managers. Application is a two-step process, so please be patient with the team. When you submit to a position on our site your information will ... View more
