61965 Data Engineer intermediate to advanced

Spring, TX - USA

Monthly Salary: Not Disclosed

Posted on: 15 hours ago

Vacancies: 1 Vacancy

Job Summary

Responsibilities:
The Senior Data Engineer will lead the design development automation and optimization of scalable cloud and on premises data infrastructure supporting enterprise applications advanced analytics and AI/ML workloads. Operating within an agile environment this role requires close collaboration with software engineering infrastructure and data science teams to deliver secure reliable and high performance data systems aligned with enterprise standards.
Responsibilities

Design build and optimize scalable ETL/ELT pipelines using Databricks Delta Lake Python SQL and related data engineering technologies.
Demonstrate expert-level proficiency in PySpark and Databricks Lakehouse architectures including schema evolution ACID transactions data quality enforcement and cost and performance optimized handling of large complex datasets-aligned with upstream telemetry SCADA-scale data and Clients high throughput processing needs.
Improve data ecosystem efficiency by implementing Lakehouse based bronze/silver/gold architectures reducing processing times through optimization techniques enforcing data quality frameworks such as Great Expectations and enhancing storage and retrieval patterns.
Collaborate with cross functional teams-data scientists engineers product owners and business stakeholders-to translate business requirements into robust technical designs supporting AI/ML enablement feature engineering real time analytics and production grade data workflows.
Strengthen CI/CD and data governance practices through automation version control testing frameworks and scalable deployment patterns for data pipelines and platform components.

Qualifications

Strong background across Azure Snowflake Delta Lake Databricks and CI/CD automation bringing the cross cloud governance minded and automation driven engineering approach required for Clients integrated enterprise scale data architecture.

Top 3 Skills:

Azure
Data Knowledge on Snowflake Delta Lake and Databricks
CI/CD

Responsibilities: The Senior Data Engineer will lead the design development automation and optimization of scalable cloud and on premises data infrastructure supporting enterprise applications advanced analytics and AI/ML workloads. Operating within an agile environment this role requires close coll...

Design build and optimize scalable ETL/ELT pipelines using Databricks Delta Lake Python SQL and related data engineering technologies.
Demonstrate expert-level proficiency in PySpark and Databricks Lakehouse architectures including schema evolution ACID transactions data quality enforcement and cost and performance optimized handling of large complex datasets-aligned with upstream telemetry SCADA-scale data and Clients high throughput processing needs.
Improve data ecosystem efficiency by implementing Lakehouse based bronze/silver/gold architectures reducing processing times through optimization techniques enforcing data quality frameworks such as Great Expectations and enhancing storage and retrieval patterns.
Collaborate with cross functional teams-data scientists engineers product owners and business stakeholders-to translate business requirements into robust technical designs supporting AI/ML enablement feature engineering real time analytics and production grade data workflows.
Strengthen CI/CD and data governance practices through automation version control testing frameworks and scalable deployment patterns for data pipelines and platform components.

Qualifications

Strong background across Azure Snowflake Delta Lake Databricks and CI/CD automation bringing the cross cloud governance minded and automation driven engineering approach required for Clients integrated enterprise scale data architecture.

Top 3 Skills: