Data Integration Engineer
Share
Job Location:
Sunnyvale, CA - USA
Monthly Salary:
Not Disclosed
Posted on:
14 hours ago
Vacancies:
1 Vacancy
Job Summary
Job Description
Key Qualifications
MUST HAVE:
Snowflake Data Engineering
o Design and implement enterprise-grade data pipelines using Snowflake including ingestion and transformation
o Must be strong in both Core and Semantic aspects
o Develop complex SQL transformations stored procedures and Dynamic tables inside Snowflake to enable near real-time and batch processing
o Implement Snowflake data sharing data marketplace integrations
o Engineer Snowpipe and Kafka-to-Snowflake streaming ingestion pipelines also handling high throughput event data at scale
o Optimize Snowflake cluster performance virtual warehouse sizing query profiling clustering keys
o Architecture design aspects performance tuning time travel warehouse concepts - scaling clustering micro-partitioning
o Experience with SnowSQL Snowpipe
Data Integration aspects
o Design and maintain end-to-end ETL/ELT pipelines using Apache Airflow
o Experience in building reusable parameterized data ingestion pipelines/frameworks is beneficial.
o Thorough on data quality checks
AI and Data Science
o Integrate AI/LLMs with data pipelines via Python UDFs or API callouts enabling text analytics semantic search and GEN-AI augmented workflows
o Experience with Python based frameworks scikit learn PyTorch TensorFlow
o Experience with NLP and text-mining techniques on unstructured data to identify actionable information
o Time-series forecasting anomaly detection and propensity modeling
Experience with Data Visualization aspects
Hands-on experience with writing Complex queries using Joins Self Joins Views Materialized Views Cursor also Recursive use of GROUP BY PARTITION BY functions / SQL Performance tuning
Hands-on experience with ETL and Dimensional Data Modelling Slowly Changing Dimensions (SCD Type 1 2 3)
o Good understanding of concepts like schema types table types - fact-dimension etc. like how to design a dimension vs fact design considerations factored etc.
Proficiency in Python scripting/programming using Pandas PyParsing Airflow.
o Pandas Tableau server modules Numpy Datetime Apache Airflow related modules APIs
o Data Pipeline automation
o Strong Python programming skills
Actively participating in discussions with business to understand requirements perform thorough impact analysis and provide suitable solutions.
Key Words to search in Resume
Snowflake Advanced SQL Dimensional Data Modelling (Slowly Changing Dimensions) Python AI Data Science Data Visualization
Location: Sunnyvale CA
Salary range:$80000-$140000 a Year.
